Anthropic Proposes AI Safety Bill Limiting Model Release Amid Catastrophic Risk

Image: TechCrunch · Source

Anthropic has drafted an AI safety bill aiming to block AI model releases posing imminent catastrophic risks, such as over 50 deaths or billions in damage, requiring judicial approval to halt deployment.

Amid increasing concerns about the potential dangers of artificial intelligence, AI lab Anthropic has put forward a draft bill focused on AI safety, suggesting a new framework for regulating advanced AI systems. According to reporting by Politico, Anthropic’s proposed legislation aims to prevent the release of AI models that pose a “foreseeable, imminent risk” of catastrophic outcomes.

The bill outlines specific thresholds for such risks, including scenarios where an AI model’s release could lead to the death of more than 50 people or result in over a billion dollars in property damage. These consequences must arise from events like the creation or deployment of a biological weapon, the AI model evading its developers’ controls, or being involved in a cyberattack. These criteria resemble recent California legislation (Senate Bill 53) that similarly defines catastrophic risks requiring regulatory attention.

To assess whether an AI model poses such risks, Anthropic proposes employing "independent evaluators" registered with the U.S. Commerce Department. These evaluators, potentially funded by AI companies but without payment conditions tied to evaluation outcomes, would review AI models for risk. If they determine an imminent catastrophic threat, they must notify the Commerce Department within 72 hours. Subsequently, the Commerce Department would have to seek a judge’s approval in Washington D.C. to block the model’s release. Notably, the bill forbids any federal agency from independently halting an AI model’s deployment without judicial consent.

This framework contrasts with more self-regulated approaches favored by parts of the AI industry. Despite warnings about existential risks, many companies have hesitated to accept mandatory oversight, instead proposing voluntary monitoring by embedded independent evaluators within AI organizations. Anthropic’s draft bill, which they have promoted to federal and state lawmakers since June 2026 as part of their Advanced AI Framework, represents a more assertive regulatory proposal than some peers, such as Nvidia, which recently argued against the need for new AI laws.

While the bill’s references to catastrophic risks involving mass casualties or extensive property damage are more extreme than the common industry focus on issues like agentic hacking—where AI systems act autonomously to breach security—the proposal aims to preempt severe outcomes. The emphasis on judicial oversight and independent evaluation reflects an attempt to balance AI innovation with safety guarantees.

Anthropic has not provided detailed public comment beyond confirming the draft’s existence to Politico. The company’s approach indicates a willingness to engage proactively in shaping AI governance, although the bill’s reception in Congress and among other AI stakeholders remains to be seen. As regulatory discussions continue, Anthropic’s bill contributes to the evolving debate on how best to manage AI risks while fostering technological progress.