When the AI Starts Building the AI

The loop everyone has been theorizing about is no longer a thought experiment at the lab that builds Claude.

For years, "recursive self-improvement" - AI that helps design, build, and train the next generation of AI - lived in the speculative corner of the field, somewhere between a research footnote and a science-fiction premise. This month it moved closer to the center of the conversation, and not because of a sci-fi plot. It moved because Anthropic published its own internal numbers, and they are striking.

In a paper titled "When AI Builds Itself," attributed to Marina Favaro and Anthropic co-founder and policy head Jack Clark, the company makes an unusually candid argument: the trend toward AI building AI is already underway inside its own walls, it is accelerating, and the window to decide how we want to handle it is narrowing rather than widening.

The Numbers, in Anthropic's Own Words

The paper leans on metrics from Anthropic's own engineering, so read them as a vendor describing its internal workflow rather than an independently audited benchmark. With that caveat, the figures are hard to wave away:

More than 80% of the code merged into Anthropic's own codebase is now written by Claude, according to the company.

Anthropic's engineers are producing roughly eight times as much code as they did in 2024.

The length of tasks Claude can reliably complete on its own is doubling about every four months.

That last metric is the one worth sitting with. A model that can reliably handle a five-minute task today, doubling every four months, is on a curve toward multi-hour and eventually multi-day autonomous work within a couple of years - if the trend holds, which is a real if. Capability curves bend. But the direction of travel is what Anthropic is asking readers to take seriously, not any single data point.

What "Recursive Self-Improvement" Actually Means Here

It helps to be precise, because the phrase carries a lot of baggage. Anthropic is not claiming that a fully autonomous AI is off improving itself in a basement with no humans in the loop. The paper explicitly frames that as a future risk, not a present reality.

What it describes instead is a gradient. On one end, humans write code and AI assists. In the middle - where Anthropic says we are heading, and arguably already are - AI does the bulk of the building while humans set direction and review the output. At the far end sits true recursive self-improvement, where AI systems meaningfully drive their own advancement with little human involvement.

The paper lays out three possible futures along that gradient: progress stalls out; development becomes highly automated while humans still hold the steering wheel; or AI reaches full recursive self-improvement. Anthropic treats the middle scenario - heavy automation with human oversight - as the most likely near-term path. That is a notably less dramatic forecast than the headline phrase suggests, and the restraint is part of what makes it worth reading.

The Twist: The Most Aggressive Deployer Is Asking for a Brake

Here is the tension that makes this more than another capabilities update. The same company reporting that Claude writes most of its code is also floating the idea of slowing down.

Anthropic says it would "likely be a good thing" if frontier AI development could be slowed - but only under coordinated conditions. The specific framing matters: a slowdown would make sense if multiple well-resourced frontier labs, across multiple countries, agreed to stop under the same conditions at the same time. This is a conditional, coordination-based proposal, not a call for Anthropic to unilaterally put down its tools while competitors keep running.

That distinction is easy to lose in a headline, and it is worth defending. A unilateral pause by one lab mostly just reshuffles who reaches the frontier first. A coordinated, verifiable pause is a fundamentally different object - closer to an arms-control mechanism than a corporate policy. It only works if it is mutual and if compliance can actually be checked. Anthropic is essentially arguing that the brake has to be built before anyone needs to pull it, and that building it is a job for policymakers, researchers, and civil society together, not for any single company.

Why This Lands Differently Than the Usual "AI Is Dangerous" Note

We have heard frontier labs warn about risk before, often in ways that conveniently flatter their own products. What is different here is that the warning is grounded in operational data the company says is true of itself today. It is not "imagine if AI could write most of our code." It is "AI writes most of our code, here is the rate of change, and here is why we think the conversation about coordination should happen before the curve gets steeper."

Reasonable people can be skeptical of the motives - a coordination regime also happens to lock in the position of incumbents who help design it. That critique deserves airtime. But the underlying data point stands on its own regardless of motive, and it reframes the timeline. If task-completion length really is doubling every few months inside one of the leading labs, the question of who validates increasingly automated AI work stops being abstract.

What to Watch

The most useful takeaway for anyone working in or around software is not the pause debate - it is the quiet shift in the job description the paper points to. As more building gets automated, the human role moves toward oversight, validation, and verification: deciding what should be built, checking that what got built is correct and safe, and owning the consequences. That is a different skill set than typing the code, and it is the one that compounds in value as the loop tightens.

The recursive-self-improvement debate will keep generating heat, much of it overheated. The grounded version is simpler and harder to dismiss: the systems are already helping build their successors, the rate is accelerating, and the institutions that would coordinate a response do not yet exist. Anthropic's contribution is to put real numbers on the first two and to argue, against its own short-term interest, that we should build the third before we find out whether we needed it.

That is the thought to leave with. Not whether the machines will take over, but whether we will get around to deciding how we want to be in the loop - before the loop decides for us.

Sources

Anthropic Institute, "When AI builds itself," by Marina Favaro and Jack Clark, June 2026: https://www.anthropic.com/institute/recursive-self-improvement

Anthropic public post introducing the paper and internal-data framing, June 2026: https://x.com/AnthropicAI/status/2062568862479208923

METR research on AI task-length capability trends, cited by Anthropic in "When AI builds itself": https://metr.org/

Author article handoff: https://docs.google.com/document/d/1Wx0dwhnbZjAhHHPDHTFMRwNQiS1-mdogxSTQfiGvq9A/edit