How to Review Pull Requests Written by an AI Coding Agent

A pull request from an AI coding agent reads like it was written by your best engineer on their most confident day. Clean diff, tidy commit message, tests passing. That confidence is exactly the problem: nothing in the PR signals which lines are solid and which are a plausible-sounding guess.

When you review AI coding agent pull requests with the same habits you'd use on a human's PR, skim the diff, check the tests are green, glance at the description, you miss the failure modes that are specific to agents. They don't get tired or careless the way people do. They hallucinate a method that doesn't exist on that SDK version, quietly skip an edge case nobody asked about, or write a test that exercises the code without asserting anything meaningful. None of that shows up as a red flag in GitHub's UI. It shows up in production.

This is a review process for product and engineering leads who need to keep merging agent-written PRs at speed without letting quality quietly slide. By the end you'll have a checklist, a couple of CI gates, and a habit for interrogating the diff instead of trusting it.

Step 1: Treat the PR description as a claim, not a fact

Agents write confident summaries of their own work, and those summaries are sometimes wrong about what the diff actually does. Read the description, then read the diff independently, then compare. If they disagree, that's your first finding before you've even started reviewing logic.

Step 2: Verify every new API call and dependency actually exists

The single most common agent failure is calling a method, flag, or package that sounds right but isn't real, especially after a library's API changed between versions. Don't eyeball this, check it.

# Node: confirm every newly imported package is a real, resolved dependency
npm ls --depth=0 2>&1 | grep -i "invalid\|missing"

# Python: confirm the lockfile actually resolves the new imports
pip install -r requirements.txt --dry-run

# Quick sanity check on a specific symbol before trusting it
python -c "import <package>; print(<package>.<new_method>.__doc__)"

If a symbol doesn't resolve, don't ask the agent to "try again", ask it which version or doc page it based the call on. If it can't cite one, assume it guessed.

Step 3: Make the agent name what it didn't handle

Agents are good at the happy path and quiet about everything else. Before approving, paste this into the PR thread and require an answer before merge:

List every input, error condition, and concurrency case this diff does NOT
handle, with the specific line where it would fail. Do not say "none" unless
you can point to a test that proves it.

A real gap list is a good sign, the agent is being honest. A confident "this handles all cases" with no test evidence is the reason to slow down.

Step 4: Read the tests for assertions, not for existence

A passing test suite from an agent-written PR tells you the code ran, not that it's correct. Agents will write a test that calls the function and asserts the return value isn't null, which passes even if the logic is backwards. Open every new test file and check that each assertion encodes an actual expected value, not just "it didn't crash."

Step 5: Reorder your own reading, don't follow the diff order

Read schema and type changes first, then the business logic that depends on them, then the tests. Agents often generate files in an order that buries a risky schema change under twenty lines of formatting noise higher in the diff. git diff --stat before you open the PR tells you which files actually matter.

git diff --stat main...feature/agent-branch

Step 6: Gate agent PRs in CI, not just in your head

Add a required label and a human reviewer rule so an agent-authored PR can never self-merge on green CI alone.

# .github/CODEOWNERS
* @your-team

# Branch protection (via GitHub UI or API): require
# - status checks: build, test, dependency-resolve
# - at least 1 human approval
# - the "agent-authored" label present before merge is allowed

Label every agent PR automatically from your CI trigger so this can't be skipped by accident.

Gotchas

• Don't let a fast, tidy diff lower your guard. Agents produce clean-looking code even when the logic is wrong.
• Watch for silently dropped error handling that a human would have kept out of habit, agents optimize for the request as stated, not for defensive coding you didn't ask for.
• Large refactors from an agent are harder to review carefully than they look. If it touched more than a few files, ask for a smaller PR instead of reviewing a wall of diff.

Agent-written code earns the same scrutiny as a stranger's PR, not less, because a confident diff is not the same thing as a correct one.