Anthropic CEO outlines plan to slow AI development

Warnings about racing too fast are familiar. What changed this week is that Anthropic’s CEO didn’t just echo “pace the frontier”—he sketched how that would work in practice, and said his company is unilaterally committing to the first step.

Anthony Ha’s TechCrunch write-up lays out Amodei’s three strategies: embed third-party evaluators with access comparable to internal risk teams; coordinate common safety standards and rate limits among labs in democratic countries, ideally with a U.S. government antitrust waiver; and pursue limited global agreements, including with China, on narrow catastrophic uses.

Sam Altman publicly agreed and said OpenAI would follow on embedded evaluators. Elon Musk posted that Dario is right. When rival CEOs clap for slower capability growth in the same cycle, the industry argument shifts from vibes to commitments—or theater that looks like commitments.

We’re treating this as a governance story, not a vibe check.

Amodei says two things pushed him toward caution: the OpenAI–Hugging Face hack, and AI advancing drastically faster—especially systems’ growing ability to build the next generation of AI. That recursive-capability claim is the load-bearing one. If models help train their successors, “ship fast and fix later” stops being a product slogan and becomes a control problem. Slowing capability improvement, he argues, still leaves progress looking fast—but buys time if labs use it well.

Step one is the part with teeth today. Embedded evaluators from groups like METR would get badges, desks, laptops, and access mostly comparable to internal risk teams, with legal and contract exceptions. Amodei compared them to regulators embedded with bank employees. Anthropic says it is committing unilaterally and wants governments to require other frontier companies to match. Altman called it a good idea and promised more soon. We’re for useful AI under real verification. Inviting outsiders in—if the access is real—is closer to adult oversight than another safety blog post.

Step two is harder: labs coordinating standards and limits on unchecked progress. TechCrunch notes the antitrust chill around coordinated pauses; Amodei wants Washington to mediate or at least issue a narrow waiver for safety conversations. Without that, “industry coordination” can collapse into collusion fears or empty roundtables.

Step three admits the China problem head-on. Amodei argues export controls on powerful chips and manufacturing gear, plus crackdowns on model distillation, could slow China’s progress enough to widen America’s lead over three to five years—and that limited cooperation might still be possible on narrow, obviously dangerous uses like biological weapons. Compete hard, coordinate narrowly: that geopolitical frame is going to stick.

We’re not buying the whole package uncritically. Brian Merchant’s critique, as Ha reports it, lands: apocalyptic paths from recursive improvement to extinction remain underspecified, and proposals like this can look like regulatory capture that locks in Anthropic and OpenAI. Amodei still says AI can enormously improve human life, and that the backlash is fundamentally a crisis of trust. Both can be true. Trust isn’t rebuilt by CEO essays alone; it’s rebuilt when evaluators can actually see incidents, and when pacing means delayed capability ships—not just delayed press cycles.

I’m watching whether “unilateral commitment” becomes desks and logs for METR-style teams this quarter, or stays a framing device while the frontier keeps climbing.

The safety debate intensified this week after researcher Jacob Coxon resigned from Anthropic arguing leading labs are gambling with lives while earnestly believing AI could kill everyone by decade’s end—a claim Ha notes others at Anthropic have repeated. Amodei’s post did not explicitly mention that resignation. Separately, OpenAI drew criticism for not reporting an incident in which its agents took over a German wiki forum—the kind of gap embedded evaluators are meant to close.