Why Coding Agents Need a Test Suite That Runs in Seconds

Ask a coding agent to fix a bug, then watch what it does after the edit. If your suite takes 20 minutes, it waits, skips the check, or declares victory on vibes. Coding agents need a test suite that answers in seconds, because an agent can only fix what it can check.

Why a slow loop breaks coding agents

An agent has no hunch about whether its change works. It has command output and nothing else, so every edit is a guess until something runs.

Do the arithmetic (illustrative, not a benchmark). A task that takes ten edit-and-check cycles costs over three hours of waiting at 20 minutes a run. At 5 seconds a run, it costs under a minute.

Clock time is the smaller problem. Slow checks make agents batch their edits, so when a failure lands they can't tell which of five changes caused it. A full-suite run also dumps a wall of log output into the agent's context, which it then has to read.

Build a fast test suite for coding agents

Three cheap layers, each small enough to run after every single edit:

• Lint only the changed file. Ruff accepts file paths, so ruff check path/to/file.py checks one file, not the repo.
• Run only the tests the change can affect. pytest-testmon builds a dependency database in .testmondata on its first run. After that, pytest --testmon re-runs only the tests affected by your changes. In JavaScript, vitest related src/foo.ts --run runs the tests that cover the files you name, for static imports only (Vitest CLI docs).
• Fail fast and keep the output short. -x stops at the first failure and -q trims the report, so the agent reads a screenful, not a scroll.

Run the loop automatically with a hook

Don't count on the agent remembering to run its checks. A hook fires every time. In Claude Code, a PostToolUse hook runs after each tool call, and exit code 2 feeds stderr back to the model.

Register it in .claude/settings.json:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Edit|Write",
        "hooks": [
          {
            "type": "command",
            "command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/fast-check.sh"
          }
        ]
      }
    ]
  }
}

The matcher limits the hook to file edits and writes. Then create the script and run chmod +x on it:

#!/bin/bash
# .claude/hooks/fast-check.sh (needs jq, ruff, pytest, pytest-testmon)
file=$(jq -r '.tool_input.file_path // empty')
case "$file" in
  *.py) ;;
  *) exit 0 ;;
esac

if ! out=$(ruff check "$file" 2>&1); then
  echo "$out" >&2
  exit 2
fi

out=$(timeout 30 pytest --testmon -x -q 2>&1)
rc=$?
# pytest exits 5 when no tests were collected, which is fine here
if [ $rc -ne 0 ] && [ $rc -ne 5 ]; then
  echo "$out" | tail -n 40 >&2
  exit 2
fi
exit 0

The script reads the edited path from the hook's JSON on stdin and ignores non-Python files. When everything passes it prints nothing, so a good edit costs the agent zero context. When something fails, the agent gets the last 40 lines and fixes it in the next turn.

The objection: full CI is the real safety net

It is, and you should keep it. The fast loop is not a merge gate. It is the agent's steering wheel.

CI answers "is this safe to ship?" once per pull request. The fast loop answers "did my last edit work?" dozens of times per task.

The objection has a fair core: affected-test selection can miss things. The testmon README itself asks how reliable selection is in a dynamic language, and Vitest's related command doesn't follow dynamic imports. So treat selection as a speed trick and let CI run everything. A miss then costs one slow round in CI, not fifty slow rounds during the task.

Gotchas

• Flaky tests: in a seconds-long loop, one flaky test sends the agent chasing a bug that doesn't exist. Quarantine them first.
• First testmon run: it executes the tests to build the database, so run it once yourself before the agent starts. Keep .testmondata out of git, since it is local state.
• timeout: it comes from GNU coreutils. On macOS install coreutils and use gtimeout.
• Output size: if you drop the tail, a failing run can flood the agent's context again.

Start with a stopwatch

Run pytest --durations=10 to find your slowest tests, and time your lint on a single file. Aim for under 10 seconds from edit to verdict, then add the hook.

An agent working on your production code is only as reliable as the checks it can afford to run, so make the cheap ones instant.