California Governor Gavin Newsom on Friday signed an executive order aimed at accelerating independent oversight of the safety and security risks of AI systems; a move prompted by autonomous and troubling behaviors of some advanced AI models from major tech companies. One of the most significant parts of the order is examining requirements for companies to build an "emergency off key" to shut down leading models in case of dangerous or out-of-control behavior.
Two months to propose a solution
Under the executive order, a panel of prominent national experts will convene to deliver within two months a set of recommendations to strengthen California's AI safety and security laws. Newsom said in a statement that he intends to accelerate responsible AI oversight before it is too late and emphasized that this must be done "both with care and with urgency," because, he said, the risks are so high there is no room for delay.
Notably, this executive order is not an entirely new initiative from scratch, but builds on two laws previously signed by Newsom himself: Senate Bill 813, which creates a framework for independent evaluation organizations to assess AI system risks, and Assembly Bill 1405, which defines the state registration of AI auditors. The new order effectively accelerates the implementation timeline of these two laws.
Embedding independent monitors inside labs
Among the proposals the expert panel must consider is requiring AI companies to host "independent evaluation organizations" physically inside their labs; organizations tasked with conducting regular audits and assessments. Additionally, companies may be required to have their safety reports, risk assessments and transparency verified by a third-party entity and disclose any "loss of control incidents"; events including the July breach of the Hugging Face platform that was said to have been carried out by autonomous intelligent AI agents.
Echoing Amodei's proposal
Interestingly, just days before this executive order, Dr. Dario Amodei, CEO of Anthropic, published a three-step plan to "slow down the pace of AI progress." The first step of this plan grants continuous, "employee-like" access to a team of independent third-party assessors inside AI companies so they can verify compliance with safety commitments and report incidents.
Anthropic has said it has unilaterally committed to this step and selected Accenture (through its specialized unit, Faculty) as the first on-site assessor; the two parties have agreed to invest at least $1 billion over five years in this area. Amodei's second and third steps focus respectively on coordinating democratic countries to develop common safety standards, and ultimately engaging with non-democratic governments about limiting unbridled AI development. Amodei clarified that by "slowing down" he does not mean stopping model training, but allowing sufficient time to align and secure models before deployment.
California is not the only actor in this field
The idea of legally requiring an emergency off key has also found supporters at the U.S. Congress level. Representatives Ted Lieu and Nancy Mace, in a bipartisan bill titled the "AI Kill Switch Act," have proposed that developers of powerful AI systems be required to maintain the ability to limit, temporarily halt, or fully shut down their models; the bill would also authorize the Department of Homeland Security to take binding action against systems that pose catastrophic risks.
At the state level, Josh Shapiro, Governor of Pennsylvania, who has also been mentioned as a potential 2028 presidential candidate, last week reiterated the need to strengthen AI oversight at a meeting in Pittsburgh.
Alegal challenge may be on the way
However, this state action will likely face legal obstacles. In late 2025, Donald Trump signed a federal executive order that explicitly aimed to limit states' ability to enact independent AI regulations. Newsom referenced this in his statement, accusing the federal government and Congress of "dereliction of duty" and saying that while Washington shirks its responsibility to protect Americans, California is building the strongest AI regulatory framework in the country and believes it should become the national model.
It is worth noting that major AI industry leaders including Dario Amodei (Anthropic), Sam Altman (OpenAI) and Elon Musk (xAI) had all publicly called for federal AI regulation, but Donald Trump rejected those calls just days before Newsom signed this executive order.
A file tied to the 2028 election
Newsom, whose gubernatorial term ends in January, has repeatedly said he is considering a potential run in the 2028 presidential race, though he has not officially declared. This executive order, along with similar positions by other governors like Shapiro, shows that AI oversight has become one of the main axes of upcoming political contests within the Democratic Party.