Timeline

California agency recommends mandatory audits and standby power to cap AI self-improvement

The 85-page report, dated 16 November, proposes licensed verifiers, 48-hour incident reporting in place of 15 days and a risk-containment protocol; it does not recommend using the cap immediately.

  • Government & policy
  • Safety & alignment
  • Notable

The California Government Operations Agency published a report to Governor Gavin Newsom recommending that the legislature strengthen the state’s frontier-AI laws. The 85-page report, “AI Safety in 2026 and Beyond”, answers the executive order Newsom issued in September. Its cover is dated 16 November; CalMatters reported on the governor’s presentation of it on 18 November, which is the date used here. Everything in it is a recommendation and would need legislation.

It argues that the federal approach of “relying on voluntary company commitments rather than actual legislation is not enough”, and says the OpenAI Hugging Face incident was “one of dozens” in which agents operated outside developer control. Its recommendations cover four areas. Baseline safety, security and transparency standards would apply to all large frontier developers. Each would also face ongoing oversight by licensed independent verification organisations, building on laws passed this autumn. The report would cut the reporting window for critical safety incidents from 15 days to 48 hours, and bring in models capable of catastrophic harm even when trained with fewer than 10^26 operations, a numerical threshold it says has kept incidents like Hugging Face outside the state’s reporting rules, which began with SB 53. And each large developer would have to maintain a “Risk Containment Protocol” showing it can shut models down at several levels, verified by an outside body.

The most discussed item concerns recursive self-improvement, where AI systems automate the research that builds the next models:

The state must have an immediate and ongoing line of sight into the possibility of recursive self-improvement (RSI) with the authority to cap its development and use if needed.

— California Government Operations Agency, AI Safety in 2026 and Beyond

Developers would report metrics on how far they automate AI research, including the share of compute devoted to it. The cap is framed as standby authority: the report states “We do not recommend that the state impose a cap immediately”, and says it would be time-limited and lifted once safeguards were verified. Some commentary described this as a ban on self-improvement; the text does not.

Reactions in CalMatters came from governor-elect Xavier Becerra, who said he would move quickly with the Legislature to put “full incident reporting and real shutdown authority into law”, and from Anthropic’s chief global affairs officer, Mariano-Florentino Cuéllar, who welcomed the move towards independent evaluation. Legislators can introduce bills when they return on 7 December. OpenAI had asked Newsom a month earlier to amend SB 53 on several of the same subjects, and the state’s attorney general sued the company over the incident two days earlier.

In the commentary

What people were saying around this time — external links, from the record's commentary rail.