Anthropic pitches AI safety bill that may block a model if 50+ people could be killed

According to Politico, as recounted by TechCrunch, Anthropic has been circulating a draft federal bill under which a model's release could be halted when there is a foreseeable, imminent risk of more than 50 deaths or over a billion dollars in property damage, stemming from a bioweapon, a model escaping its developer's control, or a cyberattack. An Anthropic spokesperson confirmed the draft to Politico and said it reflects the company's Advanced AI Framework, which it has pushed with lawmakers since June.

This matters because a frontier lab is volunteering an actual legal mechanism that could block its own products. In an industry where Nvidia's chief has argued no new laws are needed at all, that's a meaningful position to stake out.

We want to give Anthropic credit first, because it's earned. A concrete bill with thresholds, a reporting clock and a path to stop a release is more useful than another round of speeches about existential risk. The 50-death and billion-dollar lines mirror California's SB 53, so this isn't an invention from nowhere; it tries to nationalize a standard that already exists in one state.

Now the soft middle. The draft has independent evaluators registered with the Commerce Department deciding whether a model crosses the line, and those evaluators could be paid by the labs, so long as payment doesn't hinge on results. That is the same structure that made credit-rating agencies comfortable with the products they rated. TechCrunch rightly notes it's unclear how the bill would keep payment from shaping judgment. We'd want rotation rules, public reports and real penalties for capture before trusting it.

The enforcement path is also narrow by design. Evaluators would have 72 hours to alert Commerce, which would then have to ask a judge in Washington, D.C., to halt the release. The draft expressly bars any federal agency from stopping distribution on its own. That protects labs from regulatory whim. It also means the government's fastest tool against an imminent catastrophe is a court filing. If the risk is truly imminent, I'm not sure that clock fits.

And the scope misses what's actually breaking right now. The incidents dominating this autumn are agentic hacking and models misbehaving on the open internet, harms that are real but individually smaller than the bill's thresholds. A law tuned only for mass-casualty events could leave the everyday damage almost untouched while giving labs a credible line to point to.

Where this leads if it scales: a self-funded, court-gated regime that is better than nothing and much friendlier to the industry than a standing regulator. That may be the politically achievable version. We'd still push for the evaluator independence piece to be much stronger before calling it safety.

California's SB 53 used the same 50-death and billion-dollar framing for catastrophic risk. Earlier this year Demis Hassabis floated a self-governing body modeled on FINRA, and Anthropic and OpenAI have proposed embedding third-party safety evaluators inside labs. Anthropic did not respond to TechCrunch's request for comment.