Review AI-Generated Code Before Merge

## TL;DR

Review AI-generated code like any other untrusted implementation: inspect the
diff, rebuild it, test the changed behavior, check security-sensitive paths, and
only merge after a reviewer can explain why the code is correct.

The useful mental model is simple: **AI can propose code, but the reviewer owns
the merge decision.** The pull request should contain enough evidence for a
human maintainer to verify the change without relying on generated confidence.
Even when an AI reviewer provides comments or suggested changes, validate that
feedback carefully and supplement it with human review before merging.

## Prerequisites & Requirements

- [ ] {"task": "Isolated checkout", "description": "You can rebuild the branch in a disposable sandbox or container and reproduce the test results"}
- [ ] {"task": "Project test commands", "description": "You know the focused tests, linters, and type checks for the touched code"}
- [ ] {"task": "Security scanning", "description": "Code scanning, secret scanning, dependency review, or local equivalents are available for risky diffs"}
- [ ] {"task": "Reviewer authority", "description": "You can request changes when the PR lacks tests, source links, or a clear rollback path"}

## Core Concepts Explained

### AI output is an implementation, not evidence

Generated code may be useful, but the explanation around it is not proof that
the change is correct. Treat claims such as "all edge cases are handled" or "no
security impact" as review prompts. Ask for the command, test, trace, design
reference, or code path that proves the claim.

### Review behavior before style

Generated diffs often look polished. Start with behavior: what input changed,
what output changed, which permissions changed, what data moves, and what can
fail. Style cleanups can wait until the reviewer understands the actual runtime
effect.

### Separate generated-size risk from feature risk

A small generated diff can change authorization logic. A large generated diff
can be mostly generated fixtures. Review risk by the code path and blast radius,
not by whether the author used an AI tool.

## Step-by-Step Implementation Guide

1. **Freeze the PR scope.** Ask the author to state which files were generated,
   which were manually edited, and what user-visible behavior should change. If
   the PR mixes unrelated refactors, generated rewrites, and feature work,
   request a smaller branch before reviewing.

2. **Preflight dependency and tooling changes.** Before running install, build,
   or test commands from the branch, inspect changes to package manifests,
   lockfiles, package-manager configuration, install scripts, container images,
   GitHub Actions, and third-party SDKs. Confirm why each new dependency or
   lifecycle script is required.

3. **Rebuild in an isolated environment.** Pull the branch into a disposable
   sandbox, container, or isolated development environment. Install
   dependencies using the repository's documented workflow with package-manager
   lifecycle scripts disabled unless the changed scripts and packages have been
   reviewed and approved, then run the focused checks for the touched package.
   Do not rely only on screenshots, generated summaries, or copied terminal
   output in the PR body.

4. **Review security-sensitive paths first.** Start with authentication,
   authorization, secrets, payments, migrations, data deletion, networking,
   deserialization, sandbox escapes, release automation, and permission changes.
   These paths get review priority because a plausible-looking generated patch
   can still change a trust boundary.

5. **Check secrets and dependency changes.** Run secret scanning or a local
   equivalent before merge. Re-check dependency diffs after installing so
   generated or refreshed lockfiles still match the reviewed dependency set.

6. **Read the diff in small slices.** Review one behavior path at a time. For
   each slice, ask: what invariant should stay true, what test proves it, what
   user data is touched, and what happens when the new code fails?

7. **Require focused tests for changed behavior.** Unit tests are usually enough
   for pure functions. Integration tests are better for permission checks,
   persistence, API clients, migrations, and concurrency. If a test would be too
   expensive, ask for a documented manual verification command.

8. **Verify AI-written comments and docs.** Generated comments can drift from the
   code they describe. Check that public docs, migration notes, and inline
   comments match the implementation and do not promise unsupported behavior.

9. **Review agent-created PRs as production code.** If an AI coding agent opened
   the pull request, inspect the final diff thoroughly before merging. Confirm
   that the agent's summary matches the files, that requested review comments
   were actually addressed, and that no unrelated generated changes slipped in.

10. **Merge only after the reviewer-owned checklist passes.** The reviewer should
   be able to summarize the change, name the highest-risk file, point to the
   verification evidence, and explain the rollback plan.

## Reviewing With Claude Code

When the diff was produced by Claude Code, you can use the same tool to add an
independent review step. The key idea from the Claude Code best-practices guide
is that a fresh context reviews better than the one that wrote the change: a
reviewer running in a separate [subagent](https://code.claude.com/docs/en/sub-agents)
context "sees only the diff and the criteria you give it, not the reasoning that
produced the change, so it evaluates the result on its own terms." The session
that implemented the work receives the gaps directly and can fix and re-review
without you copying findings between windows.

Claude Code ships a bundled `/code-review` skill that "reviews the current diff
for bugs in a fresh subagent and returns findings to the session." Run it before
treating a change as done:

```text
/code-review
```

To check a diff against your own plan or requirements instead of generic bug
hunting, write the review prompt yourself and tell Claude to delegate it. Name
the work to check, what to check it against, and what counts as a finding:

```text
Use a subagent to review the rate limiter diff against PLAN.md. Check that
every requirement is implemented, the listed edge cases have tests, and
nothing outside the task's scope changed. Report gaps, not style preferences.
```

For a reusable reviewer, define a subagent in `.claude/agents/`. Subagents are
Markdown files with YAML frontmatter where only `name` and `description` are
required; the body becomes the subagent's system prompt, and `tools` restricts
what it can do (omit it to inherit all tools). The following restricts the
reviewer to read-only tools so it cannot edit files while reviewing:

```markdown
---
name: code-reviewer
description: Reviews code for quality and best practices
tools: Read, Glob, Grep
model: sonnet
---

You are a senior reviewer. Review the current diff for:
- Correctness against the stated requirements
- Security-sensitive paths: auth, secrets, data deletion, networking
- Missing tests for changed behavior

Report only gaps that affect correctness or the stated requirements.
Provide specific line references. Do not edit files.
```

Then invoke it explicitly, for example: `Use a subagent to review this code for
security issues.` Because subagents run in their own context window with their
own allowed tools, the review does not clutter the implementing conversation.

You can also run review and verification non-interactively, which is how Claude
Code integrates into CI pipelines and pre-commit hooks. The `claude -p` flag
runs a single prompt without a session, and `--output-format json` returns
structured output a script can parse:

```bash
# Run a focused review on the current diff in CI
claude -p "Review the staged diff for security and missing tests. Report gaps only." \
  --output-format json
```

The best-practices guide is explicit that a reviewer asked to find gaps will
usually report some even when the work is sound. Tell the reviewer to flag only
gaps that affect correctness or the stated requirements, and treat the rest as
optional, to avoid over-engineering, defensive code, and tests for cases that
cannot happen.

### Review Dimensions Reference

Use these dimensions when prompting a Claude Code review subagent or running
`/code-review`. They map the source guidance onto a concrete checklist.

| Dimension | What to check | How Claude Code helps |
| --- | --- | --- |
| Independent context | Reviewer did not write the change | Run the review in a fresh subagent or a separate session so the model is not biased toward code it just wrote |
| Correctness vs. plan | Every requirement implemented; nothing out of scope changed | Prompt a subagent to review the diff against `PLAN.md` and report gaps |
| Security-sensitive paths | Auth, secrets, data deletion, networking, deserialization | Use a read-only `code-reviewer` subagent (`tools: Read, Glob, Grep`) focused on these paths |
| Verification evidence | Tests, build exit code, or a script that produces a pass/fail | "Have Claude show evidence rather than asserting success": run the check and read the result |
| Test coverage for changes | Focused tests for the changed behavior exist | Ask the reviewer to flag changed behavior that lacks a test |
| Finding discipline | Findings are real gaps, not style nits | Tell the reviewer to report only gaps affecting correctness or requirements |
| Automated gate | Review runs without a human present | `claude -p "..." --output-format json` in CI or a pre-commit hook |

## Reviewer Checklist

- [ ] {"task": "Scope is narrow", "description": "The PR changes one behavior or one coherent workflow"}
- [ ] {"task": "Generated files are identified", "description": "The author says which parts came from an AI coding tool"}
- [ ] {"task": "Dependency preflight passes", "description": "Manifest, lockfile, package-manager, and install-script changes are reviewed before install"}
- [ ] {"task": "Isolated build succeeds", "description": "The branch installs and checks from a disposable sandbox or container with lifecycle scripts disabled unless approved"}
- [ ] {"task": "Tests cover the changed behavior", "description": "Focused tests or a manual verification command are visible to reviewers"}
- [ ] {"task": "Security-sensitive paths are inspected", "description": "Auth, permissions, data movement, secrets, dependencies, and automation changes are reviewed first"}
- [ ] {"task": "Scanners are clean or triaged", "description": "Code scanning, secret scanning, and dependency alerts are resolved or explicitly accepted"}
- [ ] {"task": "Claims have evidence", "description": "Generated explanations are backed by code, tests, logs, docs, or maintainer reasoning"}
- [ ] {"task": "Rollback is understandable", "description": "The team knows how to revert or disable the change if production behavior regresses"}

## When to Block the Merge

Block or request changes when the PR:

- Adds a dependency without a reason, version pin, lockfile update, or license/security check.
- Changes authentication, authorization, payment, data deletion, or release automation without focused tests.
- Includes generated code that the author cannot explain.
- Relies on AI-written claims instead of reproducible verification.
- Moves secret handling, logging, or telemetry into a new path without privacy review.
- Makes sweeping style or architecture rewrites while claiming to fix a small bug.

## Troubleshooting

- **The PR is too large to review**: ask for a smaller PR that separates generated refactors from behavior changes.
- **CI is green but the risk still feels high**: add a targeted test around the risky path before merge.
- **The author says the AI tool verified it**: ask for the actual command output, test case, or source link a maintainer can inspect.
- **A scanner reports a warning the team accepts**: document the rationale in the PR so future reviewers can see the decision.

## Duplicate Check

This guide is intentionally about the maintainer workflow for reviewing
AI-generated pull requests before merge. Adjacent entries in the repository cover
code review tools, security scanners, and review-oriented agents, but they do not
provide a source-backed guide for the human merge decision on generated code.

## References

- Claude Code Docs: Best practices (adversarial review, verification, `/code-review`) - https://code.claude.com/docs/en/best-practices
- Claude Code Docs: Create custom subagents - https://code.claude.com/docs/en/sub-agents
- GitHub Docs: Reviewing proposed changes in a pull request - https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/reviewing-proposed-changes-in-a-pull-request
- GitHub Docs: Responsible use of GitHub Copilot code review - https://docs.github.com/en/copilot/responsible-use/code-review
- GitHub Docs: Review output from Copilot - https://docs.github.com/en/copilot/how-tos/copilot-on-github/use-copilot-agents/review-copilot-output
- GitHub Docs: About code scanning - https://docs.github.com/en/code-security/concepts/code-scanning/about-code-scanning
- GitHub Docs: About secret scanning - https://docs.github.com/en/code-security/concepts/secret-security/about-secret-scanning
- GitHub Docs: About dependency review - https://docs.github.com/en/code-security/concepts/supply-chain-security/about-dependency-review
- NIST SP 800-218: Secure Software Development Framework - https://csrc.nist.gov/pubs/sp/800/218/final