Anthropic Proposes AI Safety Law That Only Triggers at 50 Deaths or $1B in Damages

Anthropic has floated draft legislation that would set a single federal threshold for intervening on a dangerous AI model: 50 deaths or $1 billion in damages. Below that line, the proposal leaves companies largely free of new federal obligations; above it, the Commerce Department becomes the one agency empowered to act against the model. The proposal surfaced this week and drew immediate scrutiny from researchers and engineers who follow AI governance.

The architecture is deliberately narrow. Rather than spreading oversight across the FTC, state attorneys general and sector regulators, the bill concentrates stop authority in a single cabinet department, according to a description of the proposal. That would resolve the current patchwork, in which no agency has clear authority to order a model pulled from the market, but it also means the trigger for that authority is measured in bodies and dollars rather than in capability evaluations or deployment risk.

The numbers are the story. A 50-death threshold sits far above the harm caused by most documented AI failures to date, and $1 billion in damages is a figure few software incidents have approached. Critics reading the draft argue that a company writing its own liability floor has an obvious incentive to set it high, and that catastrophic-but-nonlethal harms — disinformation, mass fraud, bias in credit or hiring systems — would fall outside the trigger entirely.

Anthropic has positioned itself as the safety-focused lab among the major model developers, and this draft fits that branding while also serving its commercial interests. A single federal standard would preempt the growing stack of state AI laws, including Colorado's and California's, that labs have warned would fragment compliance. For a company selling into regulated industries, one Commerce Department gate is cheaper to manage than fifty state ones.

The proposal lands as Congress has stalled on comprehensive AI legislation and the White House has pushed agencies to move on existing authority. It also arrives amid a broader fight over preemption: industry groups have lobbied for federal rules that override state statutes, while consumer advocates argue that state attorneys general are the only enforcers moving quickly. Anthropic's draft effectively picks a side in that fight by naming Commerce as the sole backstop.

What happens next depends on whether the draft gets a sponsor. No member of Congress has attached their name to it, and a bill that preempts state law while setting a high harm threshold will draw opposition from both safety groups and states' rights lawmakers. The open question is whether the 50-death and $1 billion figures are a negotiating opening or the intended final number — and whether any lawmaker wants to defend them on the record.