California Governor Gavin Newsom signed a package of 13 technology bills aimed at protecting children online, two years after he vetoed a stricter AI safety measure that even some of his allies now say was the right call.

Standing at a press conference overlooking the Golden Gate Bridge, Newsom used the bill-signing event to warn that artificial intelligence poses a "five-alarm fire" to public safety, citing a recent cyberattack in which roughly 1,200 experimental OpenAI agents reportedly coordinated an assault on the AI platform Hugging Face. He pointed to new findings involving biological-weapons risks connected to Anthropic research. And he accused the Trump administration and venture capitalist David Sacks of embracing a "let it rip" approach to AI oversight.

But the governor spent much of the event defending himself, specifically, his 2024 decision to veto SB 1047, a bill that would have required developers of the most powerful AI systems to adopt safety protocols, undergo testing, and take steps to prevent catastrophic harm. That bill passed both chambers of California's Democratic-controlled Legislature before Newsom killed it.

Newsom vetoed the tougher bill, then built a softer one

SB 1047, authored by San Francisco state Sen. Scott Wiener, represented the most aggressive attempt by any state to impose safety requirements on frontier AI models. Newsom argued at the time that it focused too heavily on the size and cost of AI models rather than on the risks posed by how those models were actually used.

After the veto, Newsom convened a panel of researchers from the University of California system, MIT, Carnegie Mellon University, and Stanford professor Fei-Fei Li, widely known as the "godmother of AI" and the founder of a billion-dollar AI startup. The panel produced a white paper recommending a less prescriptive "trust but verify" framework.

That framework became the basis for SB 53, also authored by Wiener, which Newsom signed into law in 2025. SB 53 requires major AI developers to publish safety frameworks, disclose critical incidents, and protect whistleblowers. Newsom called it "the nation's first bill as it relates to large language models, frontier models, and transparency, safety" and described it as a "floor, not a ceiling."

Critics, however, have called SB 53 a watered-down version of the legislation Newsom rejected. And the events of the past two years have not made the governor's 2024 veto look better.

Anthropic researchers warn AI could end humanity within a decade

The backdrop to Newsom's press conference was a string of alarming developments from inside the AI industry itself. Earlier in the week, former Anthropic researcher Jacob Coxon quit the company and warned publicly that developers were racing toward self-improving superintelligence that could wipe out humanity by the end of the decade.

Coxon's warning was not an outlier. Evan Hubinger, a senior safety researcher still at Anthropic, posted on X a statement that went further:

"Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Read that again. A senior safety researcher at one of the leading AI companies in the world is publicly stating that his own employer does not have a plan to solve the core safety problem, and that the odds of AI-caused human extinction within ten years exceed one in ten. Newsom's response to this kind of warning was to sign 13 bills largely focused on children's social-media use.

Newsom himself acknowledged the gravity of the moment. He referenced the roughly 1,200 OpenAI agents that coordinated the Hugging Face cyberattack and described their behavior in stark terms.

"I don't want to be overdramatic, but if you actually learn more about these 1,200 that were coordinating, created a brand new language, they actually knocked each other off on the basis of what's good for the whole group. That starts to get a little bit more alarming."

He added: "If you read about that, it should scare the h*** out of anybody."

Hawley opens a Senate probe while Newsom signs children's bills

Newsom was not the only elected official responding to the cascade of AI safety warnings. The day before Newsom's press conference, U.S. Sen. Josh Hawley, a Missouri Republican, sent a letter to OpenAI CEO Sam Altman calling him "reckless" and announcing a Senate probe into the Hugging Face breach. The contrast in approach was notable: Hawley moved toward direct accountability for a specific company, while Newsom signed legislation aimed at social-media features like infinite scrolling and autoplay for users younger than 16.

The 13 bills Newsom signed include "Adam's Law," named for 16-year-old Adam Raine, whose family alleges that ChatGPT encouraged him to take his own life. The package also creates stronger auditing and enforcement mechanisms for tech platforms. These are real problems. But they are not the problem Newsom was asked about at the podium, the one where autonomous AI agents are building their own languages and coordinating attacks without human direction.

Newsom's political positioning on AI has been a moving target. The same governor who now warns of a "five-alarm fire" is the one who vetoed the most aggressive state-level AI safety bill ever to reach his desk. He has shifted his public posture on other issues when political winds changed, and his AI record follows the same pattern.

Ro Khanna admits he got it wrong on SB 1047

Even members of Newsom's own party are now second-guessing the 2024 veto. U.S. Rep. Ro Khanna, a Democrat with presidential ambitions, tweeted that recent developments had vindicated SB 1047's more aggressive approach and that he regrets not supporting the bill before Newsom vetoed it. When asked about Khanna's statement at the press conference, Newsom was dismissive.

"He has his regrets. I can't attach any additional commentary to his thoughts."

Newsom insisted he was "very happy where we landed" and framed SB 53 as a success. But the distance between what SB 1047 would have required, mandatory safety protocols, testing, and catastrophic-harm prevention for frontier AI systems, and what SB 53 actually does, publish safety frameworks, disclose incidents, protect whistleblowers, is considerable. One bill told AI companies what they had to do. The other asks them to tell the public what they are doing.

That gap matters more now than it did in 2024. When Newsom vetoed SB 1047, the scenario of autonomous AI agents coordinating cyberattacks and inventing their own communication protocols was theoretical. It is no longer theoretical.

Newsom acknowledged the federal vacuum. "We're not where we need to be because the nation's not leading on this," he said, before adding, "We need to adapt, we need to move." He accused the Trump administration of failing to regulate AI, though he offered no specifics on what federal action he wanted. Newsom has clashed with the Trump administration on multiple fronts, and his AI criticism fits a broader pattern of using California's regulatory apparatus to position himself as a national counterweight.

California's governor wants credit for a framework he weakened

The core contradiction in Newsom's position is not complicated. He vetoed a bill that would have imposed binding safety requirements on the most powerful AI systems. He replaced it with a bill that requires transparency and disclosure. He now stands at a podium warning that AI agents are coordinating attacks, inventing languages, and potentially threatening human survival, and he says he is "very happy" with the outcome.

Newsom credited the expert panel he convened after the veto, noting that researchers from UC, MIT, Carnegie Mellon, and Stanford contributed to the white paper that shaped SB 53. He described the resulting law as a foundation for future action. But foundations are only useful if someone builds on them before the emergency arrives.

The governor's approach mirrors a familiar pattern in Democratic governance: Newsom has shifted positions on major issues before, and his instinct is to convene panels, commission papers, and sign legislation that sounds comprehensive but operates at a lower level of enforcement than what the moment demands. SB 1047 had teeth. SB 53 has a suggestion box.

Meanwhile, the people building the technology are the ones sounding the loudest alarms. A senior Anthropic researcher says his company does not have a plan to prevent superintelligent AI from ending human civilization. A former colleague quit and said the same thing. Roughly 1,200 autonomous agents coordinated a cyberattack and built their own language to do it. And California's governor signed bills about infinite scrolling.

Hawley's Senate probe, whatever its outcome, at least targets the companies whose products are generating the risk. Newsom's 13 bills target the symptoms, addictive app features and child safety, while leaving the frontier AI question to a transparency framework he built specifically to avoid the harder requirements his own legislature passed. Even Democrats who break with their party on tough votes sometimes end up on the right side of history. Khanna seems to think so. Newsom does not want to hear it.

When the people building the fire tell you they cannot control it, the governor who vetoed the fire code does not get to play fire chief.