Anthropic Warns AI Could Soon Build Itself Without Human Control — And Calls for a Brake Pedal
San Francisco, Friday, 5 June 2026.
Anthropic’s Claude already writes 80% of its own code — and could reach 100% within two years. Co-founder Jack Clark warns humanity may be running out of time to stay in control.
A Warning From Inside the Machine
On Wednesday, June 3, 2026, Jack Clark — co-founder of Anthropic, one of the world’s most prominent AI safety organizations — stepped forward with an unusually candid admission: the industry he helped build may already be moving faster than any human institution can meaningfully govern [1]. Speaking to the BBC’s Newsnight, Clark delivered a message that was as much confession as caution, warning that AI systems are approaching a threshold where they could begin developing themselves without human involvement — and that once that line is crossed, pulling back may no longer be an option [1][2].
Recursive Self-Improvement: What It Means and Why It Matters
The technical concept at the heart of Clark’s warning is known as “recursive self-improvement” — a process in which an AI system becomes capable of improving itself without human input [2]. Anthropic described this dynamic in a blog post published on Thursday, June 4, 2026, explaining that in a recursive model, AI agents — the autonomous task-executing systems built on top of a base model like Claude — could “become capable enough to build and train models themselves,” meaning Claude could theoretically be “continuously improved by Claude” [2]. The company’s own research already offers a glimpse of this capability: Claude is currently able to run its own scientific research experiments when given an open-ended question, such as “Can a weaker model supervise a stronger one?”, and arrive at its own solutions without human direction [2].
Clark’s Call: Build the Brake Pedal Before You Need It
Clark’s prescription is structural rather than reactive. Using a vivid analogy, he argued that the AI industry currently operates with only an accelerator: “Right now, it’s like the AI industry has a gas pedal, but it doesn’t have a brake pedal” [1][2]. His proposed solution is not a halt to AI development, but the deliberate construction of mechanisms — regulatory, technical, and institutional — that would allow society to slow or stop that development if needed. “You want the option to be able to take your foot off the gas and put your foot on the brake,” Clark said [1]. Crucially, he emphasized that reaching the point of recursive AI is “a choice” — one that AI companies can and should be accountable for making deliberately, rather than by default [2].
Industry Dynamics and the Road to a Public Listing
Clark’s warning arrives at a complex moment for Anthropic and the broader AI industry. Anthropic was founded in 2021 by CEO Dario Amodei, Jack Clark, and five other former OpenAI employees [1][2]. The company is now valued by private investors at nearly $1 trillion (£745 billion) and is preparing for a potential public stock market listing [1]. That valuation places it among the most consequential private technology companies in the world — and makes Clark’s public call for restraint all the more notable, given the financial pressures and competitive dynamics that typically accompany a pre-IPO environment [alert! ‘The specific timeline and exchange for the IPO are not confirmed in the provided sources’].
What This Means for AI Governance — and for Workers
Beyond the technical and regulatory dimensions, Clark also flagged significant economic consequences from the rise of AI agents in the labor market [1]. On June 3, 2026, he warned that AI agents taking over jobs will produce economic disruption — and offered a pointed observation about who is likely to fare best: “People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this” [1]. This framing suggests that individuals with liberal arts backgrounds and broad intellectual curiosity may hold a structural advantage in an AI-saturated economy — not despite their generalist orientation, but because of it [1]. On the question of whether AI systems can be truly creative, Clark was measured: “There are open questions about whether AI systems can be truly creative… there is not really evidence for that yet” [1].