AI News
All stories
Reported

Anthropic details layered AI code review process for Claude Code, as relayed by Theo

Anthropic's engineering article describes AI review on every pull request, risk-based human approval and staged rollouts with fast rollback.

AI in Practice · · 3 min read

Anthropic's engineering post on how its Claude Code team reviews code says most of the reading now happens in machines. People are kept for the decisions that need them. The post reports that merged pull requests to Claude Code more than doubled from June to September, while the share needing a revert fell by more than 40 percent. It gives no baseline for that revert share, so the size of the drop can't be judged from the post alone. Changes landing in Anthropic's monorepo rose from about 2,500 a day in May to more than 30,000 in September. Theo, who runs the T3 channel, relayed the post and disputed parts of its guidance.

Every pull request first clears required CI. The post lists these checks as lint, type checks, tests on every platform, and a build. Then an AI review runs several agents in parallel to hunt for bugs. Separate verifier agents vote on each finding, and only findings that enough of them agree are real get posted. Survivors are tagged Important, Nit or Pre-existing. The Pre-existing tag matters because a bug the pull request didn't introduce doesn't look like the author's fault. A security review runs as its own pass and can block a merge while any of its findings or the bug review's findings remain open. Authors must either fix each finding or resolve the thread with a reason.

Approval depends on a risk classifier that reads the change and picks a tier. Low-risk changes need no human once automated checks are green. Medium changes need one person, and high-risk changes need two. The post says the change sets the tier, not the author. Some paths always need a person, including anything owners have listed as sensitive and any CI workflow file. Only authors on a trusted list get the lighter paths. A bot's approval never counts as a human's. An approval covers one specific commit, so new commits the system can't vouch for require a fresh approval.

Merged changes go through a merge queue that reruns every check against main with the change applied. That way, two pull requests that pass separately can't combine into a broken main. New behavior usually ships behind a flag that goes first to Anthropic employees, because heavy internal use often surfaces regressions before users do. Public releases reach the latest channel before stable. Flags can be turned off without a deploy. The post also admits a gap: noticing a bad merge still depends on the author, a weekly on-call rotation, and revert and incident dashboards. The team doesn't yet have circuit breakers that tighten approval when those numbers move.

The post's adoption figure needs careful reading. When the review tool entered research preview, the share of internal pull requests getting substantive review comments rose from 16 percent to 54 percent. That measures how often reviews produce comments, not how many bugs they catch or how many escape. To check noise, the team sorts each reviewer comment by how the author responded. Using git blame, the post found that about 59 percent of what the reviewer kept flagging was code that had been there since the first pass. So the reviewer was mostly surfacing old code and rarely finding new bugs. The fixes are a round cap of three, diff-only re-review after the first few passes, and no repeats of findings already posted. Theo cited the post's figures that mean reviews to pass fell from 2.08 to 1.32 after these changes.

Theo disagreed with the order of fixes. The post says that when a review misses something, the team puts the fix into a CLAUDE.md file, a skill or a new check. Theo argued, citing Lauren's guidance, that the code should be fixed first. Static analysis should come next, and instruction files should be used only when those two fail. He also said the review product is too expensive for most real-world cases. In his view, multiple models from different families, run through a local skill, can do similar work as well or better at lower cost.

The post says people remain responsible for deciding whether to build something. They review risky code such as billing, the auto-updater and CI workflows, and they review architecture that other teams will copy. It also lists reading the evidence: test output, the command that ran and what it returned, and screenshots. A person is accountable for every shipped change, whoever typed the code.

What changed

Anthropic's Claude Code team reviews code with AI review on every pull request, risk-based human approval and staged rollouts with fast rollback.

Type: Other · Applies to: Claude Code

Why it matters

Companies using coding agents at high volume may get more from layered automated checks, risk-tiered approvals and fast rollback than from slowing human review. The reported numbers come from the company's own internal data and are not independently verified.

What you can do

Check whether your pull request rules count bot approvals as human approvals, and set approval requirements by change risk.

Original source

Where we saw it

1 source

Theo - t3.gg

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works