Anthropic has drafted a federal AI safety bill that could stop the release of an AI model in certain extreme cases. The company has been sharing the proposal with lawmakers, according to a Politico report on October 16.
An Anthropic spokesperson confirmed the draft bill to Politico. TechCrunch reported on the proposal on October 20.
What the Draft Bill Would Do
The legislation describes situations where a model’s release could be blocked. These cases involve a “foreseeable, imminent risk” to public safety.
Under the draft, that risk means more than 50 people could die or more than $1 billion in property damage could occur.
The danger must come from one of three sources. These are the creation or release of a biological weapon, a model evading its developer’s controls, or a cyberattack.
These thresholds match those in California’s Senate Bill 53, which recently became law. That law also treats such scenarios as “catastrophic risk.”
How Evaluations and Enforcement Would Work
Anthropic’s bill calls for “independent evaluators” to assess whether a model poses this kind of danger. These evaluators would register with the U.S. Commerce Department.
AI companies could pay the evaluators themselves. The bill allows this as long as the “payment is not contingent on the evaluations’ results,” Politico reported.
TechCrunch noted it is not clear how the system would prevent payments from affecting results.
If evaluators find an imminent catastrophic risk, they would have 72 hours to alert the Commerce Department. The department would then need to ask a judge in Washington, D.C., to halt the model’s release.
The bill would bar any federal agency from stopping a model’s distribution on its own. A court order would be required.
“This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” an Anthropic spokesperson said.
Anthropic did not immediately respond to a request for comment from TechCrunch.
Other Industry Views on AI Rules
The proposal follows months of concern about AI safety. Recent incidents have involved AI programs that appeared to disobey human operators and hack websites on the open internet.
In September, Anthropic and OpenAI backed a voluntary system of embedded evaluators. These third-party observers would work inside AI companies to watch for signs of risk or misuse.
In July, DeepMind’s Demis Hassabis called for a self-governing body to oversee frontier AI. He suggested it could be modeled on the Financial Industry Regulatory Authority, known as FINRA.
Nvidia CEO Jensen Huang has taken a different position. In September, he argued that the AI industry does not need any new laws at all.
TechCrunch described Anthropic’s draft as going slightly further than other regulatory proposals from the industry, including Nvidia’s.
