general

Claude Code Team: Verification Loops Boost AI Coding Success by Up to 10 Points

Summarized by AI from reporting by @ClaudeDevs on X, published under our editorial policy.

Anthropic's Claude Code team reveals how verification loops — including end-to-end, property, and differential tests — improve AI-generated code quality. Internal testing shows task success improves by up to 10 points on the hardest problems, and the approach enables dozens of AI agents to work in parallel.

A screenshot of Claude's interface showing code verification tools, illustrating Claude's verification capabilities.

Key takeaways

  • Claude Code team reports about 90% of their code is written and reviewed by Claude.
  • Verification loops improve task success by up to 10 points, with the largest gains on the hardest problems.
  • End-to-end tests are the most effective single test type because they are difficult to fool and provide definitive done criteria.
  • Property tests catch edge cases agents miss by generating thousands of inputs that try to break invariants.
  • Differential tests compare old and new systems with the same inputs, effective for large-scale migrations.

Anthropic's Claude Code team has published a detailed breakdown of how verification loops enable their AI coding agent to produce higher quality code at scale. The team reports that about 90% of their code is now written and reviewed by Claude, making verification systems critical to maintaining quality.

Internal testing found that having a separate grader check Claude's work against a rubric improved task success by up to 10 points over a standard prompting loop, with the largest gains on the hardest problems. The key insight: checks that used to be expensive — tests, benchmarks, specs, and real end-to-end runs — are now cheap to request from an AI agent, because agents can generate the tests themselves.

Why Verification Loops Matter for AI Coding Agents

Verification loops are the building blocks that let teams delegate more work to Claude. Without them, a human must check everything manually, limiting the number of AI agents that can run simultaneously. With verification loops in place, dozens or hundreds of Claudes can work in parallel, dramatically increasing productivity.

The team emphasizes that verification loops also create a feedback mechanism. If an agent repeatedly fails a test that checks logs for user data, the team can create a hook or add an explicit rule in their CLAUDE.md file to prevent the issue from recurring.

Three Types of Tests Worth Investing In

The Claude Code team identifies three test types as most valuable for AI code verification, each with distinct strengths.

End-to-End Tests: The Most Effective Single Test Type

If you have to focus on one test type, the team recommends end-to-end (E2E) tests. These describe real user flows — like adding an item to a cart, checking out, and reaching a confirmation page. E2E tests are difficult to fool: either the whole feature works or it doesn't. They provide definitive done criteria and can concretely identify what's broken.

The team notes that while unit tests are faster and less flaky, it's possible to pass them and still break the feature, because both the agent and the test may not understand the full task. With E2E tests, agents can be given hands and eyes on the real running system using tools like CDP, simulators, or API harnesses.

Property Tests: Catching Edge Cases Agents Miss

Property tests focus on edge cases that humans and AI agents tend to overlook. The approach starts with writing a statement that should never fail, then using a test library — fast-check for JavaScript/TypeScript, Hypothesis for Python, proptest for Rust — to generate thousands of inputs that try to prove otherwise.

Common properties include: - Round-trip encoding: encoding X then decoding it back yields X - Invariants: things that must always be true, like "account balance can never become negative" - Idempotency: performing an action twice has the same effect as doing it once - Equivalence: optimized code produces the same result as naive but correct code

Property libraries are especially useful for rewrites when the specification has drifted from production behavior and is no longer a reliable source for verification.

Differential Tests: Using the Old System as the Oracle

Differential tests are particularly effective for large-scale migration use cases where functionality should remain unchanged. Both the old and new systems run with the same inputs, and any difference in output indicates a bug or breaking change.

The key is to use randomly generated inputs or a sample of production traffic. The team also recommends normalizing irrelevant factors like whitespace and maintaining a list of approved changes for situations where the new environment should differ from the old.

Practical Implementation: The Verify Skill

The Claude Code team recommends creating a skill called verify in a .claude/skills/ folder in your repository. This skill codifies how to start simulators or run the application, and Claude will automatically use it. Multiple verify skills can be added as needed, placed as close to the relevant code as possible.

The team emphasizes that running the app end to end matters more than tests alone. Just as a developer wouldn't ship an iOS change without trying it in a simulator, Claude should test the whole app the way a human or QA engineer would — using the Claude Chrome extension for web changes, iOS or Android MCP for mobile, computer use for desktop, or running the service locally for backend work.

Frequently asked

What types of tests does the Claude Code team recommend for AI code verification?
The team recommends end-to-end tests, property tests, and differential tests as the three most valuable types for verifying AI-generated code.
How much of the Claude Code team's code is written by Claude?
The team reports that approximately 90% of their code is written and reviewed by Claude.
How can I implement verification loops for Claude in my own project?
Create a skill called `verify` in a `.claude/skills/` folder in your repository. This skill should codify how to start simulators or run your application, and Claude will automatically use it.
What improvement did the Claude Code team see from using verification loops?
Internal testing showed that having a separate grader check Claude's work against a rubric improved task success by up to 10 points over a standard prompting loop.