The PlaybookAI Workflows

About 90% of Claude Code Is Written and Reviewed by Claude.

Here is how they verify.

John TanNovember 17, 20268 min read

On the Claude Code team, Claude writes and reviews about 90% of the code. On 16 November Addy Osmani, who works on Claude Code at Anthropic after more than 14 years at Google, published how that team checks the work. Most of it is testing advice a good engineer already knows. Two ideas are structural, cost almost nothing, and most teams running coding agents have not done either.

The first: a change must not be able to edit the check that judges it. The second: a new gate starts by blocking nothing.

How they verify, on one screen

Seven pieces. The two at the bottom are the ones nobody copies.

The pieceWhat it does
A verify skillLives in .claude/skills/ next to the code, knows how to boot the app and drive it. Claude runs it before committing
Cheap checks firstLint and types before anything that costs tokens. Generated tests then run as ordinary software
Running the appRanked above tests. End-to-end, property and differential tests in that order
Three rounds, then a humanClaude fixes what fails. A check still failing after three rounds goes to a person
A 39-item checklistEach item has a trigger readable from a diff. Usually 3 to 10 match a given change
Rules out of reachThe reviewer reads REVIEW.md from the base branch, so a change cannot edit the check that judges it
A commit stamp and gateEvery commit records how it was verified, including "none". The gate starts in observe mode and blocks nothing

Build first The last two cost nothing and need no tokens. Everything above them is ordinary good testing.

The header of a claude.dev blog post. Title: How the Claude Code team thinks about verification. Summary: How end-to-end, property and differential tests let coding agents verify their work with evidence, and what the Claude Code team does to keep its verifiers honest. Author: Addy Osmani. Category: Engineering. Published: Nov 16, 2026.
claude.dev · Nov 16The article this post is about. It follows the team's October write-up on code review.

Screenshot of How the Claude Code team thinks about verification on the claude.dev blog, captured 17 November 2026.

The check lives where the change cannot reach it

A pull request can rewrite its copy of the rules. The reviewer never reads that copy.

Addy Osmani
Addy
Osmani

If a change can modify the check that judges it, the check is broken.

Addy Osmani  ·  Addy Osmani, Claude Code, Anthropic · 16 November 2026 · claude.dev

The team tunes its reviewer with a file called REVIEW.md, which says what the reviewer should care about in that part of the code. Anthropic's Code Review reads that file from the base or default branch, never from the pull request. So a pull request can rewrite its own copy of the rules and nothing happens. The Code Review docs spell out the rest: keep the file on a protected default branch, and to change how reviews behave, merge the rules change first.

Figure 1. Where the rules are read from
The pull request can edit its copy of the rules. The reviewer never reads that copy.
BASE BRANCH REVIEW.md rules are read from here PULL REQUEST code + tests the diff is read from here REVIEW.md, edited never read Code Review one reviewer BASE BRANCH REVIEW.md rules PULL REQUEST code + tests diff Code Review REVIEW.md, edited never read The edited copy changes nothing.
How Anthropic's Code Review treats REVIEW.md, as described on claude.dev, 16 November 2026, and in the Code Review docs, read 17 November 2026. Figure by nativefirst.

This matters more with agents than with people. A person who weakens a test to get a green build knows they did it. An agent told to make the checks pass will sometimes reach the same place by a shorter road, and the diff looks tidy. If your rules, your lint config and your CI file sit in the same branch the agent is editing, the agent is marking its own homework.

The same section has a second rule worth stealing. A review belongs to a commit. If someone pushes while the review runs, the result is stale, and the article says a passing review on an out-of-date commit means nothing.

The team wrote up the reviewer itself in October, in How the Claude Code team does code review. For a company that built that layer from scratch, see Ramp's two-week code reviewer.

A new gate starts by blocking nothing

Observe mode logs what it would have stopped. Enforce comes later, and keeps an exit.

In its own repos the team stamps every commit with how it was verified. The stamp can say "none". That one detail turns "do we verify our changes?" from an opinion into a query.

On top of the stamp sits a commit gate. It can block a commit when the session that produced it never ran /code-review or /verify. It has three modes: off, observe and enforce.

Figure 2. The commit gate's three modes
Observe comes before enforce, and enforce has an exit.
OFF Nothing is checked OBSERVE Logs what it would block Read the false positives ENFORCE Blocks the commit 2 blocks, then one passes OFF Nothing is checked OBSERVE Logs what it would block Read the false positives ENFORCE Blocks the commit 2 blocks, then one passes
The Claude Code team's commit gate, as described on claude.dev, 16 November 2026. Figure by nativefirst.

Observe mode logs what the gate would have blocked, so the team can read the false positives before anyone is stopped. Enforce mode blocks the commit, with two ways out. Any commit can explain in its message why it was not verified. And after two blocks in one session, the next commit goes through, so Claude does not get stuck.

That last rule is the honest part. A gate that can trap an agent in a loop gets switched off by a tired human at the worst moment. A gate that gives way and leaves a stamp saying so stays switched on.

Verification means running the change

Tests rank below booting the app and driving it.

Osmani ranks three kinds of test for agent work. End-to-end tests first, because they are hard to fool. Property tests, which throw thousands of generated inputs at a statement that should never fail. Differential tests, which run the old system and the new one on the same inputs during a migration.

Two panels. Left, headed Unit test failure: a terminal showing FAIL checkout.test.ts and Error: Timeout after 30s, captioned Shooting blind. No step, no trace. Right, headed End-to-end test failure: a trace with Register and Add item to cart ticked, Checkout marked failed and See confirmation page not reached, beside a screenshot of a checkout page where the payment form failed to load. Below: Checkout: Place order stayed disabled because the payment form never loaded.
Anthropic's figure · Nov 16A unit test says it timed out. An end-to-end test says which step broke and shows the screen.

Figure A from the claude.dev article. The checkout shown is Anthropic's illustration, not a real run.

Then he goes further than tests: running the app end to end matters more. The team's pattern is a skill named verify, kept in .claude/skills/ close to the code it covers, that knows how to boot the app and drive it. Claude uses it before committing. We argued the same from the other side in use the agent to test your product, not to build it.

A caution from our own week: a skill only helps if the harness loads it. NVIDIA's SkillEvaluator checks that, and Garry Tan's harness evals found skill files that were never opened.

One number backs the approach. In internal testing on structured file generation, a separate grader checking Claude's work against a rubric improved task success by up to 10 points over a standard prompting loop. It is Anthropic's own test, with no task count published, so treat it as a direction.

What reaches a human

Nobody reads every change. Failure decides who gets called.

Nobody on the team reads every change. Failures decide what needs a person.

Three rounds, then stop. Claude runs the checks and tries to fix what fails. If a check still fails after three rounds, Claude stops and brings in a human.

A 39-item checklist nobody memorises. Each reliability item has a trigger that can be read from a diff, a check, and the incidents or pull requests behind it. Claude checks only the items whose trigger matches, usually 3 to 10.

One reviewer, tuned. The team does not stack a second, more expensive reviewer on the first. It tunes the first with REVIEW.md, and each round builds on the last.

Anthropic published the same lesson again the next day, about agents rather than reviewers. The advice is to fold subagents back into one loop: one clear system prompt, 4-5 thin data tools, and firm output guardrails would cover most of the use case, because most of the scaffolding was doing work the model no longer needed done for it.

Anthropic's diagram, two panels. On the left, labelled Common pattern, five layers of structure around the model: a multi-agent orchestrator on top, an intent classifier feeding a fixed five-step reasoning pipeline, then user-segment buckets and recommendation branches before an answer. On the right, labelled What we often recommend, one loop: a single agent loop holding one clear system prompt, the model, four to five thin data tools and firm output guardrails, producing the answer.
Five layers, or one loop · Nov 17The left column is what most teams build around a model. The right is what Anthropic says usually beats it.

From Anthropic's When to fold subagents into your main agent loop, 17 November 2026, as shared by Ethan Mollick.

Ethan Mollick

Rather than a sprawl of specialized subagents built by different teams, you often get better & cheaper results using one agent to do it all.

Ethan Mollick, professor at Wharton · 17 November 2026 · on X

Hold the two together and they are one rule. Do not add a layer to supervise the model when you could tune the one you have, whether the layer is a second reviewer or a fleet of subagents.

Misses become rules, with a bar. A pattern from human review comments or reverts must show up in at least two pull requests and be visible in a diff. Three agents try to disprove it. A human approves it in a pull request.

Flaky tests get fixed or deleted. Claude may not skip them, add sleeps or retry until green. If it cannot find the root cause, it deletes the test and writes down why.

The pushback is about cost

Hacker News read it as a team with unlimited tokens. They have half a point.

The article drew a long Hacker News thread on 17 November, and much of it was unimpressed. One commenter argued the piece reads like a team with unlimited tokens: a scripted browser test gives the same answer every run at close to zero marginal cost, so why pay a model to click through the app? Another asked how anyone outside Anthropic affords dozens of agents in parallel. A third said, in effect, that this is just testing.

They have a point, and the article half concedes it. It says to run cheap checks first, and that generated tests run as software without many extra tokens. The practices in this post are the ones that cost nothing: a rules file on a protected branch, a stamp, a log. None of them is new. Almost nobody applies them to an agent.

What's next

  1. Move the rules out of reach. Read review rules, lint config and CI from the default branch.
  2. Stamp every commit. One line saying how it was verified. Allow "none".
  3. Start your next gate in observe mode. Log for a week, then enforce, with an exit.

Observe first. Then enforce.

Portrait of John Tan
John Tan ↗

Founder of nativefirst.ai. Runs everything on agents and writes down what works.

Sources
1Addy Osmani, How the Claude Code team thinks about verification, claude.dev blog, 16 November 2026: the roughly 90% figure, the grader result of up to 10 points, the three test types, the verify skill, three rounds before a human, the 39-item reliability checklist, REVIEW.md read from the base branch, commit stamps, the off, observe and enforce gate, and how misses become rules.. claude.dev
2Anthropic, Claude Code documentation, Customize Code Review, read 17 November 2026: REVIEW.md is read from the pull request's base branch or the default branch, never from the pull request's own changes, and the default branch should be protected.. code.claude.com
3Addy Osmani, Morgan Westlee Lunt and Qing Wang, How the Claude Code team does code review, claude.dev blog, 15 October 2026: the team's earlier write-up of its reviewer.. claude.dev
4Addy Osmani, personal site, read 17 November 2026: Member of Technical Staff at Anthropic working on Claude Code, after more than 14 years at Google.. addyosmani.com
5Hacker News discussion of the article, 17 November 2026. Used for attributed opinion only, not as a source for facts.. news.ycombinator.com

Read next

AI is moving really fast. Keep up.

Read the Playbook