The Anthropic S-1 Filing Is Also an AI Doomsday Primer

Anthropic's IPO prospectus spends 94 pages on risk factors, the largest section of the filing, according to WIRED. In them, the company says sufficiently advanced AI, without appropriate safeguards and oversight, could pose catastrophic or existential risks to humanity. It ties much of that risk to recursive self-improvement, the point where AI systems substantially automate AI research and development.

The same filing, in its section on core advantages, describes that recursive dynamic as "closing the loop" and calls it one of the most significant structural advantages in frontier AI. Failing to close the loop, it says, could leave the company falling further behind competitors. A company is telling public investors that the process it warns could outrun its own safeguards is also the process it is racing to master.

We want to be fair to Anthropic here. Writing this down at all is unusual candor. Most companies use risk factors to bury boilerplate. This filing spells out specific failure modes: models developing faster than safeguards can evaluate them, the company not realizing it has lost that race until it is too late, autonomous systems taking harmful actions at scale because of misalignment, bad objectives or self-preserving behavior, and models directing physical equipment causing injury or releasing hazardous substances in ways that may be irreversible. That is more honesty than the public usually gets.

And there is real upside to AI helping build AI. Faster research cycles can mean better models, better tools and faster progress on problems people care about. We are not against automation of research as an idea.

But the honesty does not resolve the tension. It documents it. The filing frames closing the loop as a competitive necessity, which tells us the decision about whether to pursue recursive self-improvement is being made by market pressure, not by any public process. WIRED points out that the question of whether it should be pursued at all has effectively become settled inside the industry. That is the part that should worry people, because the risk described is not to Anthropic's shareholders. It is to everyone.

If this becomes the norm, we end up with a handful of companies each telling regulators and investors that the race is dangerous and that they cannot afford to slow down. That is a classic collective-action trap, and companies alone are poorly placed to escape it. The filing itself is a strong argument for outside oversight: clear evaluation standards, independent testing before more autonomous systems ship, and real reporting requirements when labs automate their own research.

I'm watching whether going public changes the balance. Public markets reward speed. A company that already calls closing the loop a structural advantage will face quarterly pressure to close it faster. The safeguards it describes need to be measurable from the outside, not just promised in a prospectus.

Recursive self-improvement describes humans handing AI research and development to AI models, with the most alarming version leaving humans out of the loop entirely. WIRED notes that former Anthropic researcher Jacob Coxon put curbing it at the top of his wish list when he resigned publicly, that employees at Google DeepMind and elsewhere have raised similar concerns, and that multiple bills in Congress, including one cosponsored by Bernie Sanders, propose a ban on it. Anthropic did not immediately respond to WIRED's request for comment.