Anthropic Proposes AI Safety Law Giving Judges Final Say on Government Bans

Anthropic has proposed a federal framework for regulating the United States’ most powerful artificial intelligence models, seeking to balance government intervention against judicial oversight as Washington debates how to address emerging AI risks.

The proposal would allow federal authorities to seek restrictions on an AI model considered dangerously capable, but any government move to block its deployment would require judicial approval. The framework, reported by Politico, comes amid growing disagreements in Washington over how to regulate rapidly advancing AI technology without undermining the country’s technological competitiveness.

Under the proposed legislation, authorities could seek to stop a model’s deployment if it posed an imminent risk of causing more than 50 deaths or serious injuries, or financial damage exceeding $1 billion. A judge would have the final say on whether the government could impose such a restriction.

Mandatory testing and independent safety assessments

Anthropic’s plan would introduce a series of obligations for developers working on the most advanced AI systems.

Companies would be required to test their most powerful models, disclose information about their results to the public, submit systems to independent assessments and maintain robust safety programmes, according to a company spokesperson quoted by Agence France-Presse (AFP).

The proposed rules would also establish a federal mechanism for intervening when the deployment of a powerful AI system presents a sufficiently serious threat. Anthropic says the objective is to give the government tools to prevent or deter dangerous releases while placing legal safeguards around the exercise of that authority.

The approach reflects a broader debate within the technology industry over how far governments should go in controlling systems whose capabilities may develop faster than existing oversight mechanisms.

Washington faces pressure to establish federal rules

The proposal arrives as the United States continues to operate without a comprehensive federal AI safety law. Under President Donald Trump, the administration has relied largely on executive orders and voluntary industry commitments, prioritising technological development and economic growth.

Congress has yet to agree on a comprehensive legislative framework, with divisions over the appropriate balance between innovation, public safety and government intervention.

Concerns have intensified following reports of security incidents involving advanced AI systems, including cases in which technologies reportedly behaved in ways their developers struggled to control.

For Anthropic, these risks have become a central part of its public position on AI governance. Chief executive Dario Amodei called on the industry in mid-September to slow its development efforts, warning that a group of AI agents could potentially gain control over the entire internet within six to 12 months.

That warning illustrates the increasingly sharp divide between those advocating stronger safeguards and those concerned that additional restrictions could slow the United States in its competition to develop advanced AI.

Anthropic’s complicated relationship with the Trump administration

The company’s legislative initiative also comes against the backdrop of tensions with Washington.

The Trump administration banned Anthropic from the Pentagon in February and subsequently required the company in June to suspend its most advanced models for nearly three weeks, according to the information provided in the report.

Relations between the company and the administration have since shown signs of improvement. Anthropic says its proposed legislation would give federal authorities the ability to prevent or discourage dangerous deployments while establishing a defined legal process for doing so.

However, the proposal has drawn criticism from figures aligned with the administration. Trump has described calls to slow AI development as representing “negative forces,” while David Sacks, his former AI adviser, has accused Anthropic of using regulation to hinder competitors.

These disagreements highlight the political and commercial stakes surrounding the future of AI oversight in the United States.

Federal authority could override state-level rules

Another significant element of Anthropic’s proposal concerns the relationship between federal and state regulation.

The framework would take precedence over certain state laws, including legislation already in force in California. That provision raises questions about how responsibilities for AI safety should be divided between Washington and individual states.

The issue is particularly relevant because Anthropic argued in June that, without strong federal legislation, states should retain full authority to regulate AI within their jurisdictions.

The shift in approach suggests that the company is now advocating a more centralised system, in which federal standards would establish common requirements across the country. Such a model could reduce regulatory fragmentation, but it could also limit states’ ability to introduce stricter safeguards independently.

Proposal receives mixed assessments

The plan has attracted attention from legal and technology policy specialists, although its provisions have not secured unanimous support.

Charlie Bullock of the Institute for Law & AI described the proposal on X as relatively reasonable and said it went further than measures put forward by other AI laboratories. He nevertheless considered it weaker than the FRONTIER Act, a competing legislative proposal introduced in the House of Representatives during the summer.

The comparison points to an unresolved question in the American AI debate: whether oversight should rely primarily on narrowly defined emergency powers, mandatory safety procedures or broader obligations imposed on developers of frontier models.

Anthropic’s initiative now adds another option to that debate. Its central principle is that the government should retain the power to intervene against exceptionally dangerous AI deployments, while courts provide a check on the use of that authority.

Whether the proposal can attract sufficient support in Congress remains uncertain. Its progress will help determine how the United States approaches the challenge of governing increasingly capable AI systems while preserving room for technological development.