Anthropic Urges California to Grant Courts Model Shutdown Powers

Anthropic has petitioned California Governor Gavin Newsom to grant state courts the explicit statutory authority to order shutdowns of frontier artificial intelligence models.

The frontier developer submitted recommendations urging the administration to establish legal mechanisms that mandate external safety audits and judicial kill switches. Under the proposed framework, judges could compel AI firms to halt the operation or fine-tuning of systems that present unmitigated catastrophic threats.

The submission focuses on models trained using vast computational resources, where standard internal evaluations may fail to identify dangerous autonomous capabilities. Anthropic argued that self-regulation cannot substitute for enforceable legal boundaries. The company advised that developers should be legally obligated to demonstrate that robust safety mitigations are functional before deploying systems at scale.

Mandatory external audits form the second pillar of the proposal. Anthropic recommended that licensed, independent third parties assess a lab’s safety and security protocols prior to model release. These auditors would review red-teaming evaluations, cyber-defence postures, and safeguards designed to prevent the proliferation of biological or cybernetic hazards. If an audit reveals critical vulnerabilities, regulators could seek court intervention before release.

The petition also calls for clear technical standards regarding shutdown capabilities. Rather than relying on informal operational procedures, developers would need to engineer verifiable fail-safes into their deployment pipelines. These controls must allow operators—or regulatory authorities acting through court orders—to terminate active model instances rapidly if hazardous behaviours emerge.

The move marks a departure from standard Silicon Valley lobbying playbooks, which typically resist pre-deployment oversight and state-level enforcement powers. While several trade associations have warned that mandatory audit regimes and judicial shutdown authority would impede domestic innovation, Anthropic maintained that clear statutory rules reduce regulatory ambiguity for responsible developers.

Governor Newsom has faced conflicting pressure from technology leaders, civil society groups, and researchers over how California should regulate frontier systems. With several high-compute models scheduled for release in coming quarters, state officials must decide whether voluntary industry agreements provide sufficient public protection.

Can court-ordered kill switches offer meaningful protection if frontier models become widely distributed across unmonitored infrastructure?