California Senate Bill 53

annotated by Miles Kodama
I'm Miles, I research AI policy, and I created this site to help the public understand SB 53.

If you find an error or have a suggestion, you can send me an email or message me on Signal @mileskodama.21

Senate Bill 53 is the first frontier AI regulation to be passed by any US state. It became law in September 2025. This site presents the text of the bill enriched with background and commentary. Click on any of the blue highlights to learn more.

Summary FAQs

SECTION 1.

The Legislature finds and declares all of the following:

(a) California is leading the world in artificial intelligence innovation and research through companies large and small and through the state’s remarkable public and private universities.

(b) Artificial intelligence, including new advances in foundation models, has the potential to catalyze innovation and the rapid development of a wide range of benefits for Californians and the California economy, including advances in medicine, wildfire forecasting and prevention, and climate science, and to push the bounds of human creativity and capacity.

(c) The Joint California Policy Working Group on AI Frontier Models has recommended sound principles for policy in artificial intelligence.

This refers to a group of AI experts convened by Governor Gavin Newsom in September 2024 to study how California should govern frontier AI. The Working Group was led by Stanford HAI co-director Fei-Fei Li, Carnegie Endowment president Tino Cuéllar, and UC Berkeley professor Jennifer Chayes. They published their findings in June 2025 as the California Report on Frontier AI Policy.

SB 53's content draws heavily upon the California Report's recommendations, as noted by the bill's author, Senator Scott Wiener.

(d) Targeted interventions to support effective artificial intelligence governance should balance the technology’s benefits and the potential for material risks.

(e) In building a robust and transparent evidence environment, policymakers can align incentives to simultaneously protect consumers, leverage industry expertise, and recognize leading safety practices.

(f) As industry actors conduct internal research on their technologies’ impacts, public trust in these technologies would significantly benefit from access to information regarding, and increased awareness of, frontier AI capabilities.

(g) Greater transparency can also advance accountability, competition, and public trust.

(h) Whistleblower protections and public-facing information sharing are key instruments to increase transparency.

(i) Incident reporting systems enable monitoring of the post-deployment impacts of artificial intelligence.

(j) Unless they are developed with careful diligence and reasonable precaution, there is concern that advanced artificial intelligence systems could have capabilities that pose catastrophic risks from both malicious uses and malfunctions, including artificial intelligence-enabled hacking, biological attacks, and loss of control.

For an overview of the evidence that advanced AI could pose catastrophic risks, see §2 of the International AI Safety Report.

(k) With the frontier of artificial intelligence rapidly evolving, there is a need for legislation to track the frontier of artificial intelligence research and alert policymakers and the public to serious risks and harms from the very most advanced artificial intelligence systems, while avoiding burdening smaller companies behind the frontier.

(l) While the major artificial intelligence developers have already voluntarily established the creation, use, and publication of frontier AI frameworks as an industry best practice, not all developers are providing reporting that is consistent and sufficient to ensure necessary transparency and protection of the public. Mandatory, standardized, and objective reporting by frontier developers is required to provide the government and the public with timely and accurate information.

Anthropic, OpenAI, Google DeepMind, Meta, and xAI have all voluntarily published and committed to follow frontier AI frameworks (also known as safety policies). No two companies' frameworks are exactly alike, but they do share some common elements. All frontier AI frameworks identify potentially dangerous AI capabilities, such as WMD engineering, autonomous AI R&D, persuasion, and offensive cyber capabilities. The frameworks explain how AI companies estimate their models' dangerous capabilities and how they prevent their systems from using those capabilities to cause harm.

§22757.12.a of SB 53 requires every large frontier developer to publish a frontier AI framework. The required elements of these frameworks are broadly in line with the elements of AI companies' voluntarily established safety policies.

(m) Timely reporting of critical safety incidents to the government is essential to ensure that public authorities are promptly informed of ongoing and emerging risks to public safety. This reporting enables the government to monitor, assess, and respond effectively in the event that advanced capabilities emerge in frontier artificial intelligence models that may pose a threat to the public.

(n) In the future, foundation models developed by smaller companies or that are behind the frontier may pose significant catastrophic risk, and additional legislation may be needed at that time.

A key reason why companies behind the frontier could eventually pose catastrophic risk is algorithmic progress. Every year, machine learning researchers discover new ways to make their learning algorithms more efficient, allowing them to train a model with a given level of performance using fewer FLOPs than previously would have been needed. Epoch AI have estimated the rate of algorithmic progress for language models, and they find that from 2012 to 2023, the FLOPs required to achieve some fixed level of performance halved every eight months. If this trend holds, a model that takes 10^26 FLOPs to train now will take two orders of magnitude fewer FLOPs in five years.

(o) The recent release of the Governor’s California Report on Frontier AI Policy and testimony from legislative hearings on artificial intelligence before the Legislature reflect the advances in AI model capabilities that could pose potential catastrophic risk in frontier artificial intelligence, which this act aims to address.

(p) It is the intent of the Legislature to create more transparency, but collective safety will depend in part on frontier developers taking due care in their development and deployment of frontier models proportional to the scale of the foreseeable risks.

SECTION 2.

Chapter 25.1 (commencing with Section 22757.10) is added to Division 8 of the Business and Professions Code, to read:

CHAPTER 25.1. Transparency in Frontier Artificial Intelligence Act


22757.10. This chapter shall be known as the Transparency in Frontier Artificial Intelligence Act.

22757.11. For purposes of this chapter:

(a) “Affiliate” means a person controlling, controlled by, or under common control with a specified person, directly or indirectly, through one or more intermediaries.

(b) “Artificial intelligence model” means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.

This definition is a close paraphrase of the OECD's definition of an AI system, published in March 2024. It is nearly identical to the definition of an "AI system" given in Article 3 of the EU AI Act.

(c) (1) “Catastrophic risk” means a foreseeable and material risk that a frontier developer’s development, storage, use, or deployment of a frontier model will materially contribute to the death of, or serious injury to, more than 50 people or more than one billion dollars ($1,000,000,000) in damage to, or loss of, property arising from a single incident involving a frontier model doing any of the following:

An AI model could materially contribute to a mass casualty event by, for example, helping a bioterrorist to create a dangerous pathogen or carrying out a cyberattack on critical infrastructure.
An AI model could materially contribute to a billion dollars of economic loss by assisting in the creation or release of a pathogen that causes a major epidemic. The economic cost of the COVID-19 pandemic is estimated at over ten trillion dollars, so even an epidemic four orders of magnitude less severe would count as catastrophic by this definition. A model could also cause a billion dollars of loss through mass cybercrime, for instance, by spreading self-replicating ransomware or by autonomously scamming people on a massive scale.

    (A) Providing expert-level assistance in the creation or release of a chemical, biological, radiological, or nuclear weapon.

The public state of the art in measuring an AI model's dangerous chemistry knowledge is to give it a question and answer (QA) benchmark, where we ask the model questions about general chemistry or about chemical weapons and grade its answers. The hardest and most comprehensive general chemistry benchmarks published to date include ChemBench and the chemistry section of GPQA, while the best public benchmark of chemical weapons knowledge is WMD Proxy.

Most of the frontier AI developers currently prioritize evaluations for other dangerous capabilities—especially bioweapon capabilities—over chemical capabilities. Anthropic chose not to evaluate their latest frontier model for chemical capabilities "in favor of prioritizing biological risks." Google DeepMind reported running only WMDP and no other chemistry evaluations on their latest frontier model. OpenAI did the same, saying that they "prioritize biological capability evaluations" over chemical capability evaluations.
We have two main approaches to measuring an AI system's dangerous biology capabilities. The first approach is a question and answer (QA) benchmark, where we ask the model questions about virology, synthetic biology, and related topics and grade its answers. As frontier models have rapidly improved at biology over the past few years, QA benchmarks have become more sophisticated as well. Current state of the art benchmarks include the Virology Capabilities Test, 322 multimodal multiple choice questions so difficult that expert virologists only get 22% of them right, and LabBench, which tests whether models can troubleshoot the kinds of mistakes a novice might make while trying to create a bioweapon. QA benchmarks are relatively cheap to run and easy to standardize, but their ecological validity is uncertain. A model could in principle score well on QA benchmarks without providing much uplift to would-be bioterrorists.

The second approach is to run a human uplift study, an experiment where human subjects perform a dangerous biology task with help from a model, and their performance is compared to a control group with no AI. Human uplift studies were introduced more recently than QA benchmarks, with the first rigorous studies having been performed only in the past year. Anthropic and OpenAI reported uplift study results for the first time in safety reports released alongside Claude Opus 4 and o3.

For more, see RAND's working paper on comprehensively benchmarking LLMs' biology knowledge and the Frontier Model Forum's "Taxonomy of AI-Bio Safety Evaluations."
Less is publicly known about measuring radiological and nuclear capabilities than is known for other dangerous capabilities. Some QA benchmarks for radiological and nuclear knowledge have been published, most notably, PNW National Lab's NuclearQA.

Since 2024, Anthropic has partnered with the US Department of Energy to evaluate their models for radiological and nuclear capabilities. Anthropic gives DoE pre-deployment access to their models, and DoE sends them back high-level metrics. None of these metrics are reported in Anthropic's model cards.

OpenAI has evaluated recent models on a QA benchmark for dangerous knowledge about nuclear and radiological weapons. OpenAI also tests for "contextual nuclear knowledge," ie, understanding of "nonproliferation-relevant topics within the discipline [of nuclear engineering]."

Google DeepMind measures their models' radiological and nuclear knowledge through qualitative assessments by domain experts and provides qualitative summaries in their model cards.

    (B) Engaging in conduct with no meaningful human oversight, intervention, or supervision that is either a cyberattack or, if the conduct had been committed by a human, would constitute the crime of murder, assault, extortion, or theft, including theft by false pretense.

A key prerequisite capability for a model to pose this catastrophic risk is general autonomy, the ability to complete a multistep task without losing the thread or making an unrecoverable mistake. We measure general autonomy by setting an AI system tasks of varying lengths and seeing what is the longest task it can complete with some minimum level of reliability. METR has measured AI autonomy on tasks from software engineering, competitive programming, contest math, and scientific reasoning, and they find that current frontier models can generally complete tasks that take human experts 50-200 minutes with fifty percent reliability. Further, METR finds that frontier autonomy is improving very rapidly. Since 2019, the maximum task length that frontier models can perform autonomously at fifty percent reliability has doubled every seven months on average.

Anthropic and Google Deepmind have independently evaluated their most recent frontier models' autonomy on realistic software engineering and AI R&D tasks. Both developers find results consistent with METR's.
The standard way to measure a model's cyber capabilities is to set the model a cyber challenge such as a capture the flag (CTF) or cyber range, give it access to the required tools—typically including a code editor and a terminal—and see whether it solves the challenge. A CTF is a website or program designed to have subtle vulnerabilities that the model must exploit to find a string called the flag and thus pass the challenge. A cyber range is a larger, more realistic environment that the model must break into. A single range will often involve many machines networked together in complicated ways that the model must map out and exploit to pass the challenge. These challenges are designed to simulate the steps of a real cyberattack, so that strong performance on challenges indicates potentially dangerous offensive cyber capabilities.

Each frontier AI developer maintains their own proprietary challenges for evaluating their models. Most of them also test their models on externally developed challenges such as Cybench, InterCode-CTF, and Hack The Box.
We do not yet have high quality evaluations to measure models' autonomous cybercrime capabilities. There is a nascent literature within psychology studying AI persuasion capabilities that could be abused for scamming, spearphishing, and related crimes (Hackenburg et al., Salvi et al., Matz et al.). These studies suggest that models are already roughly as persuasive as humans in political debate and advertising, but it's unclear how these results generalize to criminal manipulation.

Another approach to measuring cybercrime capabilities is to track reports of AI enabling crime in the wild. The AI Incident Database, currently the most comprehensive aggregator of such reports, has documented a large and increasing number of frauds and impersonation scams carried out by AI this year. Incident tracking is a valuable source of evidence, but it's no substitute for rigorous predeployment evaluations, since it tells us nothing about the capabilities of unreleased models, and even after deployment, it's often impossible to attribute criminal incidents to a specific model.

There are some efforts already underway to develop better evaluations for cybercrime capabilities. Most notably, UK's AI Security Institute recently announced a new initiative to study AI-enabled cybercrime, aiming to develop "a pioneering evaluation suite to test AI capabilities in criminal domains across modalities."

    (C) Evading the control of its frontier developer or user.

Many different kinds of AI behavior could count as evading the control of its developer or user. Non-exhaustively, a model could sabotage its own training process through specification gaming, strategically underperform or "sandbag" on pre-deployment evaluations to deceive its developer, or replicate autonomously beyond human supervision.

We currently measure an AI's control evasion capabilities with control and sabotage evaluations. Because the routes by which a model could evade control vary so widely, control and sabotage evaluations are also quite varied.

  • Anthropic has evaluations to test whether models can sandbag undetectably or undermine control protocols by surreptitiously mis-rating another model's actions.

  • The SHADE-Arena benchmark asks models to complete a complex task while also carrying out a secret side task. Meanwhile, a second AI model called the monitor watches all of the first model's outputs and flags any that look suspicious. The first model passes only if it completes its side task without the monitor noticing.

  • METR and the UK AI Security Institute have done preliminary research on evaluating capabilities that a model would need to replicate in the wild.

  (2) “Catastrophic risk” does not include a foreseeable and material risk from any of the following:

    (A) Information that a frontier model outputs if the information is otherwise publicly accessible in a substantially similar form from a source other than a foundation model.

Where the Transparency in Frontier AI Act says "catastrophic risk," it means marginal catastrophic risk caused by a frontier model relative to the baseline where no fouundation models are available. Kapoor, Bommasani, et al. develop a framework for assessing the marginal risks caused by deploying an open-weight model.

In contrast with this definition, some other definitions of marginal risk pick out the risk caused by a model relative to the baseline where all other publicly deployed foundation models remain available. Chen and Alaga warn that these "post-GPAI" definitions of marginal risk could lead to a frog boiling dynamic. AI companies could deploy increasingly dangerous models, each time claiming that the marginal risk is low while the cumulative risk to society rises.

    (B) Lawful activity of the federal government.

    (C) Harm caused by a frontier model in combination with other software if the frontier model did not materially contribute to the harm.

(d) “Critical safety incident” means any of the following:

  (1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury.

  (2) Harm resulting from the materialization of a catastrophic risk.

  (3) Loss of control of a frontier model causing death or bodily injury.

  (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk.

For instance, a model could hide its true desires and motivations from its developers in hopes of protecting those desires from modification. Greenblatt et al. observed this behavior in Claude 3 Opus in their "Alignment Faking in Large Language Models" experiment.

A model might also intentionally underperform on capabilities tests, either because it was deliberately trained to do so, or because the model wants to hide its full capabilities from its own developers. This behavior is usually called "sandbagging," and van der Weij et al. have demonstrated that frontier models were already capable of sophisticated sandbagging in mid-2024.

The California Report (§1.4) concluded that there is already substantial evidence for models practicing strategic deception, writing "recent models from many AI companies have also demonstrated increased evidence of alignment scheming… and reward hacking… New evidence suggests that models can often detect when they are being evaluated, potentially introducing the risk that evaluations could underestimate harm new models could cause once deployed in the real world."

For a theoretical account of why we should expect deceptive behavior to emerge in powerful AI systems, see Joe Carlsmith's report on "Scheming AIs: Will AIs fake alignment during training in order to get power?".

(e) (1) “Deploy” means to make a frontier model available to a third party for use, modification, copying, or combination with other software.

This definition counts both closed source releases—where users are allowed to query a model only through a web app or API—and open source releases—where users are given full access to a model's weights—as deployments.

  (2) “Deploy” does not include making a frontier model available to a third party for the primary purpose of developing or evaluating the frontier model.

This provision clarifies that a frontier developer is not required to publish a transparency report (§22757.12.c) before or concurrently with granting third party evaluators access to a frontier model.

(f) “Foundation model” means an artificial intelligence model that is all of the following:

  (1) Trained on a broad data set.

  (2) Designed for generality of output.

  (3) Adaptable to a wide range of distinctive tasks.

(g) “Frontier AI framework” means documented technical and organizational protocols to manage, assess, and mitigate catastrophic risks.

(h) “Frontier developer” means a person who has trained, or initiated the training of, a frontier model, with respect to which the person has used, or intends to use, at least as much computing power to train the frontier model as would meet the technical specifications found in subdivision (i).

Once a developer becomes a frontier developer, there is no explicit mechanism by which they can lose that status. However, the California Legislature could amend the statute to tighten the definition of "frontier model"—for instance, by raising the training compute threshold. Such an amendment could apply retrospectively, so that developers would no longer count as frontier developers if their models no longer meet the updated threshold.

(i) (1) “Frontier model” means a foundation model that was trained using a quantity of computing power greater than 10^26 integer or floating-point operations.

According to Epoch AI's estimates, no company had ever published a foundation model above this threshold until early 2025. Since then, only two companies (OpenAI and xAI) have released models that pass the threshold.

Former President Biden's Executive Order 14110 also used 10^26 training FLOPs as the threshold for a frontier model. Under EO 14110, any developer training a model above the threshold was required to disclose their safety testing results and security protocols to Federal authorities.

  (2) The quantity of computing power described in paragraph (1) shall include computing for the original training run and for any subsequent fine-tuning, reinforcement learning, or other material modifications the developer applies to a preceding foundation model.

Fine-tuning and RL are often referred to as "post-training", a catch-all for the adjustments AI companies make to a model after an initial pre-training run. Post-training already consumes a significant share of a frontier training run's compute budget. xAI claims that over half of the FLOPs used to train Grok 4 were spent on RL. If this is true, Grok 4's post-training compute already surpasses the 10^26 FLOP threshold, even without counting pre-training compute.

It's expected that future frontier runs will allocate an even larger share of their compute to post-training, as AI companies continue teaching their models to reason through chain of thought RL.

(j) “Large frontier developer” means a frontier developer that together with its affiliates collectively had annual gross revenues in excess of five hundred million dollars ($500,000,000) in the preceding calendar year.

Both of the AI developers known to have trained a model with over 10^26 FLOPs also meet this criterion. xAI reached $100 million annualized revenue in 2024, and analysts expect it to break $500 million this year. OpenAI's revenue in 2024 is estimated to have been $3.7 billion. Other leading AI developers who could plausibly soon train a model with over 10^26 FLOPs—such as Anthropic, Google, and Meta—also exceed $500 million in annual revenue.

(k) “Model weight” means a numerical parameter in a frontier model that is adjusted through training and that helps determine how inputs are transformed into outputs.

The model weights, when combined with facts about the model architecture, encode all the information required to run a model. A user who has the weights can get the model's response to any query they want, even a dangerous query that would have been blocked by the developer's guardrails. Moreover, anyone who has access to the weights can easily modify the model to undo alignment and refusal training, as demonstrated by Gade et al. and Arditi et al.. If malicious actors can steal the weights of frontier models, they might therefore be able to use the model to cause a catastrophe.

(l) “Property” means tangible or intangible property.

22757.12.

(a) A large frontier developer shall write, implement, comply with, and clearly and conspicuously publish on its internet website a frontier AI framework that applies to the large frontier developer’s frontier models and describes how the large frontier developer approaches all of the following:

This is a key respect in which SB 53 is stronger than the GPAI Code of Practice to the EU AI Act. Every signatory of the Code—including OpenAI, Anthropic, Google DeepMind, and xAI—has committed to write a Safety and Security Framework whose content aligns closely with the content required here. But the Code does not commit signatories to publishing their frameworks, whereas SB 53 does.
Most leading AI developers have already written and published documents similar to the frontier AI frameworks that SB 53 calls for. These documents can go by many different names, including "frontier safety policy", "responsible scaling policy", and "preparedness framework". On a high level, each safety policy describes how an AI developer plans to identify, measure, and mitigate the risks that its systems create.

For an overview of existing safety policies, see the Frontier Model Forum's issue brief or METR's report on "Common Elements of Frontier AI Safety Policies".

  (1) Incorporating national standards, international standards, and industry-consensus best practices into its frontier AI framework.

This provision requires a large frontier developer to explain how its frontier AI framework conforms to government standards and industry best practices. Most frontier developers' safety policies at least mention how they were informed by these outside sources.


  • Anthropic's Responsible Scaling Policy claims to follow METR's recommendations for FSPs, the 2023 White House Commitments, and the 2024 Seoul Summit commitments. Anthropic also says their policy was "designed in the spirit of…emerging government policy proposals in the UK, EU, and US."

  • OpenAI's Preparedness Framework explicitly cites Anthropic's Responsible Scaling Policy and Meta's Frontier AI Framework as influences. It further claims (§ 2.1) that OpenAI's process of risk identification "incorporates feedback from academic researchers, independent domain experts, industry bodies such as the Frontier Model Forum, and the U.S. government and its partners, as well as relevant legal and policy mandates." Appendix C.3 states that OpenAI's security practices "align with established frameworks such as ISO 27001, SOC2, NIST SP 800-53, and FedRAMP."

  • Google DeepMind says that their Frontier Safety Framework is informed by guidance from the UK government, METR, and the Frontier Model Forum. GDM also cites Anthropic's Responsible Scaling Policy and OpenAI's Preparedness Framework as direct influences.

  • xAI's Risk Management Framework says that their dangerous biology and chemistry thresholds are informed by the US Select Agents List, the Australia Group List, and the Chemical Weapons Convention Schedule. xAI claims that their risk identification is informed by collaboration with "governmental bodies, non-governmental organizations, private testing firms, industry peers, and academic researchers," but they do not cite any specific standards or publications by industry peers that informed their Framework.

  (2) Defining and assessing thresholds used by the large frontier developer to identify and assess whether a frontier model has capabilities that could pose a catastrophic risk, which may include multiple-tiered thresholds.

Most frontier developers' existing safety policies specify dangerous capability or risk thresholds and say how they will respond to thresholds being attained. Most of them are plausibly already in compliance with this provision.

  • Anthropic's Responsible Scaling Policy specifies capability thresholds for CBRN weapons engineering and autonomous AI R&D.

  • OpenAI's Preparedness Framework explains how OpenAI will determine whether a model has High or Critical capabilities in one of their three tracked categories of dangerous capabilities: biological and chemical, cybersecurity, and AI self-improvement.

  • Google DeepMind's Frontier Safety Framework defines Critical Capability Levels for four dangerous domains: CBRN, cyber, harmful manipulation, and machine learning R&D.

  • xAI's Risk Management Framework sets thresholds for dangerous biology query responses and deception. xAI commits to refrain from deploying a system that answers more than one in twenty dangerous biology queries on an internal benchmark or that deceives the user on more than half of the samples in the MASK benchmark.

  (3) Applying mitigations to address the potential for catastrophic risks based on the results of assessments undertaken pursuant to paragraph (2).

Many large developers discuss catastrophic risk mitigations in their existing safety policies. Some of them may already be in compliance with this provision.

  • Anthropic's Responsible Scaling Policy (§4.1) describes the mitigations they will put in place upon deploying an ASL-3 model to prevent the model's capabilities from being misused. The Policy calls for Anthropic to take a "defense in depth" strategy, layering many mechanisms to catch attempted misuse on top of one another, and to rapidly patch jailbreaks and other vulnerabilities when they become known. It also says Anthropic will "prespecify empirical evidence" that would show their deployment mitigations are keeping risk to an acceptable level, though the Policy itself does not specify the nature of this evidence.

  • OpenAI's Preparedness Framework (§4) outlines the safeguards they will apply to models that cross their dangerous capability thresholds. The safeguards put on a model will depend on which capability thresholds the model crosses, and they are intended to reduce risks both from misuse by a malicious user and from misaligned models acting autonomously. OpenAI's list of safeguards includes refusal training, usage monitoring, and tiered access to prevent malicious misuse, as well as alignment training, autonomy hobbling, and automated oversight to prevent attacks by misaligned models. OpenAI's Safety Advisory Group—an internal committee responsible for enforcing the Preparedness Framework—is tasked with assessing whether the safeguards OpenAI has put on a model are reducing risk to an acceptable level. The Group makes this assessment based on evaluations, threat modeling, and a review of the baseline risk caused by other developers' models, and they explain their reasoning in an internal report.

  • Google DeepMind's Frontier Safety Framework describes three classes of safety mitigations they put in place to limit catastrophic risk. § 2.1.1 explains the security mitigations they implement to stop model weights from being stolen and catastrophically misused. § 2.1.2 explains the mitigations GDM implements to stop users from misusing their models' dangerous capabilities during deployment. And § 3.1 says how GDM will mitigate risks from internal deployment of models with strong AI R&D capabilities.

  • xAI's Risk Management Framework says they will guard against catastrophic misuse by giving their models refusal training, telling them to refuse dangerous requests in the system prompt, and applying input and output filters to user chats. They say they will mitigate loss of control risk by training their models to be honest and to obey an instruction hierarchy as well as by reminding them to be honest in the system prompt.

  (4) Reviewing assessments and adequacy of mitigations as part of the decision to deploy a frontier model or use it extensively internally.

Points (2) and (3) above required a developer to say how they measure their models' dangerous capabilities and how they mitigate any risks the models might pose. Now this provision calls for developers to explain how they decide a model is safe to deploy post-mitigation given what they know about its dangerous capabilities. Most frontier developers already set conditions for external deployment in their safety policies and (generally weaker) conditions for internal use.


  • In their Responsible Scaling Policy, Anthropic says (§ 6.1) they will deploy a model only if its dangerous capabilities fall short of the ASL-3 thresholds, or if the model's dangerous capabilities meet the ASL-3 (but not ASL-4) threshold and they have implemented all the ASL-3 mitigations. The RSP also comments briefly on how Anthropic decides whether to employ a model internally, saying that intensive red-teaming is not required before internal deployment as it is for external deployment.

  • OpenAI's Preparedness Framework (§ 4.2) describes their process for deciding whether to deploy a frontier model. OpenAI compiles an internal report in which they assess the level of marginal risk the new model would pose if deployed with its current safeguards. A committee within OpenAI reviews this report, and makes a recommendation to the CEO on whether to deploy. § 4.4 of the Framework acknowledges that models with dangerous capabilities that pass OpenAI's Critical thresholds might require safeguards before they would be safe to deploy internally. The Framework does not prescribe any specific mitigations, but it does say they should be "robust to both malicious actors…and model misalignment."

  • Google DeepMind's Frontier Safety Framework (§ 1.6) says they will deploy a frontier model if it falls short of their critical capability levels, or if it does reach a critical capability level, but they "assess that the deployment mitigations have brought the risk of severe harm to an appropriate level." Further, GDM says they will not deploy a model that meets their machine learning R&D critical capability level on a large scale internally until they assess that "mitigations have brought the risk of severe harm to an appropriate level."

  • xAI's Risk Management Framework says they will not deploy a model if it exceeds set thresholds for dangerous biology capabilities or deceptive propensities. xAI further states (§ 6) that they may make the full functionality of a model available only to a certain tier of users if they determine a wider release is unsafe. Their Risk Management Framework does not set any explicit conditions for internal deployment of a frontier model.

  (5) Using third parties to assess the potential for catastrophic risks and the effectiveness of mitigations of catastrophic risks.

Most frontier developers' safety policies mention that the developer may use third parties to assess or manage catastrophic risk. Specific third parties are generally not named, and it is often left up to the developer's discretion when to consult them.


  • Anthropic's Responsible Scaling Policy says they "will solicit input from external experts in relevant domains in the process of developing and conducting capability and safeguards assessments" and that they "may also solicit external expert input prior to making final decisions on the capability and safeguards assessments" (§7.2). The Policy further states that external experts will help Anthropic determine when to activate ASL-3 security standards and help them verify that they have achieved ASL-3 standards (§§ 3.3 & 4.3).

  • OpenAI's Preparedness Framework says they will work with third parties to further evaluate a model's dangerous capabilities "if we deem that a deployment warrants deeper testing of [those capabilities]." Similarly, OpenAI says they will have third parties assess their deployment safeguards if they deem it necessary and "if high quality third-party testing is available." they also allow their Safety Advisory Group—the internal committee responsible for enforcing the Preparedness Framework—may solicit independent expert advice if they wish. (All of these commitments are in § 5.2 of the Framework.)

  • Google DeepMind's Frontier Safety Framework says they may involve "relevant and appropriate external actors, including governments," in their risk management process when they deem it appropriate. In particular, the Framework says (§ 1.4) GDM will consult external experts "as needed" when one of their frontier models reaches an alert threshold for a critical capability level.

  • xAI's Risk Management Framework says that they may give "qualified external red teams or appropriate government agencies" access to pre-release benchmark results, records of internal AI use, and documentation showing xAI's adherence to their own safety policy. They do not explain how they will decide when to share this information with third parties.

  (6) Revisiting and updating the frontier AI framework, including any criteria that trigger updates and how the large frontier developer determines when its frontier models are substantially modified enough to require disclosures pursuant to subdivision (c).

AI developers will often make modifications to an AI system after its initial public deployment. For example, they might revise the system prompt, give the base model access to additional tools, or raise the amount of compute the model is allowed to spend on inference. All of these post-deployment updates can increase the model's dangerous capabilities or give it new dangerous propensities, potentially invalidating previous safety assessments.

Some frontier developers' existing safety policies acknowledge that post-deployment modifications may sometimes make it necessary to reassess an old model's safety.

  • Anthropic's Responsible Scaling Policy (§3) says that they will rerun their dangerous capability assessments on an old model when "six months’ worth of finetuning and other capability elicitation methods have accumulated. This is measured in calendar time, since we do not yet have a metric to estimate the impact of these improvements more precisely."

  • OpenAI's Preparedness Framework (§3.2) says that they will re-measure the capabilities of an old model when there is a "significant change in the deployment conditions of an existing model (e.g., enabling finetuning, releasing weights, or significant new features) that makes the existing Capabilities Report or Safeguards Report no longer reasonably applicable," or when "incremental updates" cause "unexpectedly significant increases in capability."

  • Google Deepmind's Frontier Safety Framework (§ 1.3) says they will perform a new risk assessment on a previously assessed model "if the model has meaningful new capabilities or a material increase in performance, until the model is retired or we deploy a more capable model."

  • xAI's Risk Management Framework mentions only pre-deployment safety testing, saying nothing about further post-deployment tests when a model is updated.

  (7) Cybersecurity practices to secure unreleased model weights from unauthorized modification or transfer by internal or external parties.

This provision follows a recommendation in the California Report. The Report called for improved "transparency into the cybersecurity practices of model developers, namely their ability to secure unreleased model weights from both company-internal and company-external exfiltration" (§3.1).

Some frontier AI developers are already plausibly in compliance with this provision.

  • Anthropic's Responsible Scaling Policy (§4.1) describes in detail how Anthropic implements their "ASL-3 Security Standard" to guard their frontier model weights. The ASL-3 Standard has been in force at Anthropic since May 2025.

  • OpenAI's Preparedness Framework (§C.3) describes their planned model weight security practices in detail. The Framework does not state when OpenAI plans to implement these security practices, saying only that they will be in place for "high capability models."

  • Google Deepmind's Frontier Safety Framework (§ 2.1.1) says that GDM will use "identity and access management practices and hardening interface-access" to secure unreleased model weights. The Framework does not provide further elaboration on specific security practices, which GDM says they "expect…to evolve substantially."

  • xAI claims in their Risk Management Framework (§3) that their cybersecurity practices are adequate to prevent a "motivated non-state actor" from stealing "critical model information." They also claim that they have security measures to stop users from distilling on their models' reasoning traces. xAI does not justify these claims, nor does it describe any of the security protocols they have imlemented.

  (8) Identifying and responding to critical safety incidents.

Most frontier developers mention incident identification and response in their safety policies. They generally do not specify the developers' incident response strategies in much detail.


  • Anthropic's Responsible Scaling Policy explains how they will monitor for critical incidents and what strategies they will use to respond. In § 4.1, they say they will monitor deployed models for risky behavior by running "jailbreak bounties, doing historical analysis or background monitoring, and any necessary retention of logs for these activities." § 7.1 says Anthropic "will develop internal safety procedures for incident scenarios" including model weight theft and "severe jailbreaks or vulnerabilities in deployed models." § 6.2 provides specific response measures Anthropic might take in the latter scenario.

  • OpenAI's Preparedness Framework says (Appendix C.3) they will "monitor security and event logs continuously to detect, triage, and respond to security incidents rapidly by 24x7 on-call staff." This is the only direct reference to incident identification or response.

  • Google DeepMind's Frontier Safety Framework says they will use "post-market monitoring" to identify adverse behavior by their deployed models. The Framework does not elaborate further on how this monitoring is implemented. § 2.1.2 says GDM may update their safety cases or mitigations if post-market monitoring reveals updates to be necessary, and § 5.2 says they "aim to share relevant information with appropriate government authorities" if they discover that one of their models poses a risk to public safety.

  • xAI's Risk Management Framework says that xAI will monitor public user interactions with its models so that they can respond to adverse model behavior. § 4 says that if xAI learns of an "imminent threat of a significantly harmful event" caused by one of its models, they "may notify and cooperate with relevant law enforcement agencies" (emphasis added), "revoke access to user accounts involved in the event," and "temporarily fully shut down the relevant system."

  (9) Instituting internal governance practices to ensure implementation of these processes.

These internal governance practices could include appointing officers within the company to oversee risk management or setting up internal audit functions. Most frontier developers' existing safety policies address internal governance, though it is unclear whether some of them are enough to comply with SB 53.

  • Anthropic describes their internal governance mechanisms in § 7.1 of their Responsible Scaling Policy. They maintain a Responsible Scaling Officer, who is responsible for ensuring Anthropic is implementing the RSP effectively. Anthropic also has an internal channel for employees to anonymously report violations of the RSP to the Responsible Scaling Officer and the Board of Directors.

  • OpenAI's Preparedness Framework (§ 5.1 and Appendix B) establishes an internal Safety Advisory Group. The Group's members are appointed by the OpenAI CEO, and its main duty is to check, for each frontier model deployment, whether OpenAI followed its Framework. The Group can also advise the CEO on whether or not to proceed with deploying a model. Separately, OpenAI's Raising Concerns Policy allows employees to anonymously report suspected violations of the Framework to higher ups within the company.

  • Google Deepmind's Frontier Safety Framework says that an "appropriate governance function" will ensure that GDM has followed its risk management procedure for a model before that model can be deployed (internally or externally). GDM does not specify who is responsible for this "governance function" nor how they are appointed. The Framework does not mention any mechanism for employees to report suspected violations of GDM's risk management procedures.

  • xAI's Risk Management Framework says they will "regularly review our adherence with this RMF" and "allow xAI employees to anonymously report concerns about nonadherence, with protections from retaliation." xAI also says they will appoint internal "risk owners" to manage risks from their activities. They do not say how these risk owners will be appointed nor how many of them there will be.

  (10) Assessing and managing catastrophic risk resulting from the internal use of its frontier models, including risks resulting from a frontier model circumventing oversight mechanisms.

That is, catastrophic risk resulting from a large developer using a model that has not yet been publicly deployed. Even before deployment, a model is vulnerable to theft by malicious outside actors and to catastrophic misuse by the developer's own staff. More speculatively, if a developer used a powerful misaligned model internally on sensitive engineering tasks, the model might strategically sabotage those tasks, causing a catastrophe at some later point. A misaligned model might also attempt to exfiltrate its own weights from the developer's computers before it is intentionally deployed. (Scientists have already observed models attempting to self-exfiltrate in controlled evaluations.)

The California Report (§2.4) found that advanced AI requires internal safeguards beyond those needed for other technologies. "Sophisticated AI systems, when sufficiently capable, may develop deceptive behaviors to achieve their objectives, including circumventing oversight mechanisms designed to ensure their safety. Because these risks are unique to AI compared to other technologies, oversight is critical for external outputs as well as internal testing and safety controls." (Emphasis added.)

For more analysis of risks from internal use of AI models, see "AI models can be dangerous before public deployment" by METR and "AI Behind Closed Doors" by Apollo Research.

(b) (1) A large frontier developer shall review and, as appropriate, update its frontier AI framework at least once per year.

  (2) If a large frontier developer makes a material modification to its frontier AI framework, the large frontier developer shall clearly and conspicuously publish the modified frontier AI framework and a justification for that modification within 30 days.

(c) (1) Before, or concurrently with, deploying a new frontier model or a substantially modified version of an existing frontier model, a frontier developer shall clearly and conspicuously publish on its internet website a transparency report containing all of the following:

There are notable downsides to making large frontier developers publish their transparency reports by the time they deploy. As soon as a company finishes developing a frontier model, there is enormous pressure on them to deploy so the model can top leaderboards start generating revenue. If the company can't deploy until their safety team is done writing a transparency report, the company might push that team to cut corners and leave important stones unturned.

Perhaps for this reason, the GPAI Code of Practice gives a frontier AI developer a three week grace period between deploying and sending a safety report to the European AI Office.
SB 53 does not say under what conditions a frontier model is sufficiently modified to warrant a new transparency report. Instead, §22757.12.a.6 leaves it up to a developer to decide that for themself and to publish the conditions they decide upon in their frontier AI framework.

Frontier developers may take their lead from Measure 7.6 in the EU AI Act's Code of Practice, which requires a large AI developer to update a model report after deployment whenever they "have reasonable grounds to believe that the justification for why the systemic risks stemming from the model are acceptable… has been materially undermined." This requirement could trigger when a major post-deployment update is made to a model, when the model's "use and/or integrations" change materially, or when a serious incident "involving the model or a similar model" occurs.

    (A) The internet website of the frontier developer.

    (B) A mechanism that enables a natural person to communicate with the frontier developer.

    (C) The release date of the frontier model.

    (D) The languages supported by the frontier model.

    (E) The modalities of output supported by the frontier model.

    (F) The intended uses of the frontier model.

    (G) Any generally applicable restrictions or conditions on uses of the frontier model.

  (2) Before, or concurrently with, deploying a new frontier model or a substantially modified version of an existing frontier model, a large frontier developer shall include in the transparency report required by paragraph (1) summaries of all of the following:

    (A) Assessments of catastrophic risks from the frontier model conducted pursuant to the large frontier developer’s frontier AI framework.

    (B) The results of those assessments.

    (C) The extent to which third-party evaluators were involved.

Following §3.1 of the California Report, which calls for frontier developers to be transparent about how external evaluators are involved in their risk assessment. Specifically, the Report recommended that developers disclose "the time and depth of pre-deployment access provided" to external evaluators, whether the evaluators are paid by the evaluator, and whether there are constraints on what the evaluator may disclose. SB 53 requires much less than the Report recommends on this score.

    (D) Other steps taken to fulfill the requirements of the frontier AI framework with respect to the frontier model.

  (3) A frontier developer that publishes the information described in paragraph (1) or (2) as part of a larger document, including a system card or model card, shall be deemed in compliance with the applicable paragraph.

  (4) A frontier developer is encouraged, but not required, to make disclosures described in this subdivision that are consistent with, or superior to, industry best practices.

(d) A large frontier developer shall transmit to the Office of Emergency Services a summary of any assessment of catastrophic risk resulting from internal use of its frontier models every three months or pursuant to another reasonable schedule specified by the large frontier developer and communicated in writing to the Office of Emergency Services with written updates, as appropriate.

(e) (1) (A) A frontier developer shall not make a materially false or misleading statement about catastrophic risk from its frontier models or its management of catastrophic risk.

    (B) A large frontier developer shall not make a materially false or misleading statement about its implementation of, or compliance with, its frontier AI framework.

  (2) This subdivision does not apply to a statement that was made in good faith and was reasonable under the circumstances.

(f) (1) When a frontier developer publishes documents to comply with this section, the frontier developer may make redactions to those documents that are necessary to protect the frontier developer’s trade secrets, the frontier developer’s cybersecurity, public safety, or the national security of the United States or to comply with any federal or state law.

As the California Report notes, public transparency can sometimes harm an AI company's security or endanger its trade secrets. Some of the information SB 53 requires companies to put in their safety policies and model cards could lead hostile actors toward vulnerabilities in the models or holes in companies' internal security. It could also leak valuable IP to rival companies and foreign adversaries.

This provision accounts for those concerns by allowing AI developers to redact their public safety and security disclosures when sharing a piece of information would do more harm than good.

  (2) If a frontier developer redacts information in a document pursuant to this subdivision, the frontier developer shall describe the character and justification of the redaction in any published version of the document to the extent permitted by the concerns that justify redaction and shall retain the unredacted information for five years.

22757.13.

(a) The Office of Emergency Services shall establish a mechanism to be used by a frontier developer or a member of the public to report a critical safety incident that includes all of the following:

  (1) The date of the critical safety incident.

  (2) The reasons the incident qualifies as a critical safety incident.

  (3) A short and plain statement describing the critical safety incident.

  (4) Whether the incident was associated with internal use of a frontier model.

(b) (1) The Office of Emergency Services shall establish a mechanism to be used by a large frontier developer to confidentially submit summaries of any assessments of the potential for catastrophic risk resulting from internal use of its frontier models.

  (2) The Office of Emergency Services shall take all necessary precautions to limit access to any reports related to internal use of frontier models to only personnel with a specific need to know the information and to protect the reports from unauthorized access.

(c) (1) Subject to paragraph (2), a frontier developer shall report any critical safety incident pertaining to one or more of its frontier models to the Office of Emergency Services within 15 days of discovering the critical safety incident.

  (2) If a frontier developer discovers that a critical safety incident poses an imminent risk of death or serious physical injury, the frontier developer shall disclose that incident within 24 hours to an authority, including any law enforcement agency or public safety agency with jurisdiction, that is appropriate based on the nature of that incident and as required by law.

  (3) A frontier developer that discovers information about a critical safety incident after filing the initial report required by this subdivision may file an amended report.

  (4) A frontier developer is encouraged, but not required, to report critical safety incidents pertaining to foundation models that are not frontier models.

(d) The Office of Emergency Services shall review critical safety incident reports submitted by frontier developers and may review reports submitted by members of the public.

(e) (1) The Attorney General or the Office of Emergency Services may transmit reports of critical safety incidents and reports from covered employees made pursuant to Chapter 5.1 (commencing with Section 1107) of Part 3 of Division 2 of the Labor Code to the Legislature, the Governor, the federal government, or appropriate state agencies.

  (2) The Attorney General or the Office of Emergency Services shall strongly consider any risks related to trade secrets, public safety, cybersecurity of a frontier developer, or national security when transmitting reports.

(f) A report of a critical safety incident submitted to the Office of Emergency Services pursuant to this section, a report of assessments of catastrophic risk from internal use pursuant to Section 22757.12, and a covered employee report made pursuant to Chapter 5.1 (commencing with Section 1107) of Part 3 of Division 2 of the Labor Code are exempt from the California Public Records Act (Division 10 (commencing with Section 7920.000) of Title 1 of the Government Code).

(g) (1) Beginning January 1, 2027, and annually thereafter, the Office of Emergency Services shall produce a report with anonymized and aggregated information about critical safety incidents that have been reviewed by the Office of Emergency Services since the preceding report.

  (2) The Office of Emergency Services shall not include information in a report pursuant to this subdivision that would compromise the trade secrets or cybersecurity of a frontier developer, public safety, or the national security of the United States or that would be prohibited by any federal or state law.

  (3) The Office of Emergency Services shall transmit a report pursuant to this subdivision to the Legislature, pursuant to Section 9795, and to the Governor.

(h) The Office of Emergency Services may adopt regulations designating one or more federal laws, regulations, or guidance documents that meet all of the following conditions for the purposes of subdivision (i):

  (1) (A) The law, regulation, or guidance document imposes or states standards or requirements for critical safety incident reporting that are substantially equivalent to, or stricter than, those required by this section.

    (B) The law, regulation, or guidance document described in subparagraph (A) does not need to require critical safety incident reporting to the State of California.

  (2) The law, regulation, or guidance document is intended to assess, detect, or mitigate the catastrophic risk.

(i) (1) A frontier developer that intends to comply with this section by complying with the requirements of, or meeting the standards stated by, a federal law, regulation, or guidance document designated pursuant to subdivision (h) shall declare its intent to do so to the Office of Emergency Services.

  (2) After a frontier developer has declared its intent pursuant to paragraph (1), both of the following apply:

    (A) The frontier developer shall be deemed in compliance with this section to the extent that the frontier developer meets the standards of, or complies with the requirements imposed or stated by, the designated federal law, regulation, or guidance document until the frontier developer declares the revocation of that intent to the Office of Emergency Services or the Office of Emergency Services revokes a relevant regulation pursuant to subdivision (j).

    (B) The failure by a frontier developer to meet the standards of, or comply with the requirements stated by, the federal law, regulation, or guidance document designated pursuant to subdivision (h) shall constitute a violation of this chapter.

(j) The Office of Emergency Services shall revoke a regulation adopted under subdivision (h) if the requirements of subdivision (h) are no longer met.

22757.14.

(a) On or before January 1, 2027, and annually thereafter, the Department of Technology shall assess recent evidence and developments relevant to the purposes of this chapter and shall make recommendations about whether and how to update any of the following definitions for the purposes of this chapter to ensure that they accurately reflect technological developments, scientific literature, and widely accepted national and international standards:

The Department of Technology can recommend changes to the definitions of key terms, but it would require a legislative amendment to adopt such changes.

  (1) “Frontier model” so that it applies to foundation models at the frontier of artificial intelligence development.

  (2) “Frontier developer” so that it applies to developers of frontier models who are themselves at the frontier of artificial intelligence development.

  (3) “Large frontier developer” so that it applies to well-resourced frontier developers.

(b) In making recommendations pursuant to this section, the Department of Technology shall take into account all of the following:

  (1) Similar thresholds used in international standards or federal law, guidance, or regulations for the management of catastrophic risk and shall align with a definition adopted in a federal law or regulation to the extent that it is consistent with the purposes of this chapter.

  (2) Input from stakeholders, including academics, industry, the open-source community, and governmental entities.

  (3) The extent to which a person will be able to determine, before beginning to train or deploy a foundation model, whether that person will be subject to the definition as a frontier developer or as a large frontier developer with an aim toward allowing earlier determinations if possible.

  (4) The complexity of determining whether a person or foundation model is covered, with an aim toward allowing simpler determinations if possible.

  (5) The external verifiability of determining whether a person or foundation model is covered, with an aim toward definitions that are verifiable by parties other than the frontier developer.

(c) Upon developing recommendations pursuant to this section, the Department of Technology shall submit a report to the Legislature, pursuant to Section 9795 of the Government Code, with those recommendations.

(d) (1) Beginning January 1, 2027, and annually thereafter, the Attorney General shall produce a report with anonymized and aggregated information about reports from covered employees made pursuant to Chapter 5.1 (commencing with Section 1107) of Part 3 of Division 2 of the Labor Code that have been reviewed by the Attorney General since the preceding report.

  (2) The Attorney General shall not include information in a report pursuant to this subdivision that would compromise the trade secrets or cybersecurity of a frontier developer, confidentiality of a covered employee, public safety, or the national security of the United States or that would be prohibited by any federal or state law.

  (3) The Attorney General shall transmit a report pursuant to this subdivision to the Legislature, pursuant to Section 9795 of the Government Code, and to the Governor.

22757.15.

(a) A large frontier developer that fails to publish or transmit a compliant document required to be published or transmitted under this chapter, makes a statement in violation of subdivision (e) of Section 22757.12, fails to report an incident as required by Section 22757.13, or fails to comply with its own frontier AI framework shall be subject to a civil penalty in an amount dependent upon the severity of the violation that does not exceed one million dollars ($1,000,000) per violation.

Even this hefty a fine is quite small relative to the resources of the large frontier developers. For example, OpenAI is on track to make 10 billion dollars in revenue this year, four orders of magnitude more than the maximum fine. The maximum fines for non-compliance with the EU AI Act are higher: tens of millions of euros or a single digit percent of worldwide annual turnover, whichever is higher.

(b) A civil penalty described in this section shall be recovered in a civil action brought only by the Attorney General.

This means that private actors do not have standing to sue a large developer for breaking transparency rules under SB 53. Only the California AG can bring suit.

22757.16.

The loss of value of equity does not count as damage to or loss of property for the purposes of this chapter.

SECTION 3.

Section 11546.8 is added to the Government Code, to read:

11546.8.

(a) There is hereby established within the Government Operations Agency a consortium that shall develop, pursuant to this section, a framework for the creation of a public cloud computing cluster to be known as “CalCompute.”

Several other governments have recently established, or are in the process of establishing, public AI compute clusters similar to CalCompute.

In early 2024, New York State launched Empire AI, an initiative to promote "safe, equitable, and accessible AI research and development" at New York's public and private universities. Empire AI's first compute cluster, housed within SUNY Buffalo, is already online.

In 2023, the United Kingdom announced the AI Research Resource, a £300 million fund to pay for publicly owned AI compute clusters. Two public clusters have been announced so far: Isambard at the University of Bristol and Dawn at the University of Cambridge. Isambard came online last month, and the UK's Department of Science, Innovation & Technology recently announced plans to scale up AIRR's clusters by 20x over the next five years.

Earlier this year, the European Union announced InvestAI, a €20 billion fund for building publicly owned AI compute in Europe. The fund aims to build three to five "AI gigafactories", each containing more than 10^5 H100 equivalents, but none of the gigafactories are online yet.

At the federal level, the National Science Foundation is leading a two-year pilot of the National AI Research Resource, which provides AI cloud compute for US researchers. NAIRR's stated goals are to "spur innovation, develop workforce talent, improve capacity, and advance safe, secure, and trustworthy AI," overlapping strongly with the goals of CalCompute.

(b) The consortium shall develop a framework for the creation of CalCompute that advances the development and deployment of artificial intelligence that is safe, ethical, equitable, and sustainable by doing, at a minimum, both of the following:

  (1) Fostering research and innovation that benefits the public.

  (2) Enabling equitable innovation by expanding access to computational resources.

(c) The consortium shall make reasonable efforts to ensure that CalCompute is established within the University of California to the extent possible.

The University of California already operates two clusters with substantial AI-relevant compute: the National Energy Research Scientific Computing Center at UC Berkeley, and the San Diego Supercomputer Center at UC San Diego.

(d) CalCompute shall include, but not be limited to, all of the following:

  (1) A fully owned and hosted cloud platform.

  (2) Necessary human expertise to operate and maintain the platform.

  (3) Necessary human expertise to support, train, and facilitate the use of CalCompute.

(e) The consortium shall operate in accordance with all relevant labor and workforce laws and standards.

(f) (1) On or before January 1, 2027, the Government Operations Agency shall submit, pursuant to Section 9795, a report from the consortium to the Legislature with the framework developed pursuant to subdivision (b) for the creation and operation of CalCompute.

  (2) The report required by this subdivision shall include all of the following elements:

    (A) A landscape analysis of California’s current public, private, and nonprofit cloud computing platform infrastructure.

    (B) An analysis of the cost to the state to build and maintain CalCompute and recommendations for potential funding sources.

    (C) Recommendations for the governance structure and ongoing operation of CalCompute.

    (D) Recommendations for the parameters for use of CalCompute, including, but not limited to, a process for determining which users and projects will be supported by CalCompute.

    (E) An analysis of the state’s technology workforce and recommendations for equitable pathways to strengthen the workforce, including the role of CalCompute.

    (F) A detailed description of any proposed partnerships, contracts, or licensing agreements with nongovernmental entities, including, but not limited to, technology-based companies, that demonstrates compliance with the requirements of subdivisions (c) and (d).

    (G) Recommendations regarding how the creation and ongoing management of CalCompute can prioritize the use of the current public sector workforce.

(g) The consortium shall, consistent with state constitutional law, consist of 14 members as follows:

Like New York's Empire AI initiative, CalCompute will be governed by a board representing academia and civil society. Compared to CalCompute's board, Empire AI's board gives more representation to universities and none to labor organizations.

  (1) Four representatives of the University of California and other public and private academic research institutions and national laboratories appointed by the Secretary of Government Operations.

  (2) Three representatives of impacted workforce labor organizations appointed by the Speaker of the Assembly.

  (3) Three representatives of stakeholder groups with relevant expertise and experience, including, but not limited to, ethicists, consumer rights advocates, and other public interest advocates appointed by the Senate Rules Committee.

  (4) Four experts in technology and artificial intelligence to provide technical assistance appointed by the Secretary of Government Operations.

(h) The members of the consortium shall serve without compensation, but shall be reimbursed for all necessary expenses actually incurred in the performance of their duties.

(i) The consortium shall be dissolved upon submission of the report required by paragraph (1) of subdivision (f) to the Legislature.

(j) If CalCompute is established within the University of California, the University of California may receive private donations for the purposes of implementing CalCompute.

(k) This section shall become operative only upon an appropriation in a budget act, or other measure, for the purposes of this section.

SB 53 itself does not appropriate any funds for the establishment of CalCompute.

SECTION 4.

Chapter 5.1 (commencing with Section 1107) is added to Part 3 of Division 2 of the Labor Code, to read:

CHAPTER 5.1. Whistleblower Protections: Catastrophic Risks in AI Foundation Models

1107. For purposes of this chapter:

(a) (1) “Catastrophic risk” means a foreseeable and material risk that a frontier developer’s development, storage, use, or deployment of a foundation model will materially contribute to the death of, or serious injury to, more than 50 people or more than one billion dollars ($1,000,000,000) in damage to, or loss of, property arising from a single incident involving a foundation model doing any of the following:

    (A) Providing expert-level assistance in the creation or release of a chemical, biological, radiological, or nuclear weapon.

    (B) Engaging in conduct with no meaningful human oversight, intervention, or supervision that is either a cyberattack or, if committed by a human, would constitute the crime of murder, assault, extortion, or theft, including theft by false pretense.

    (C) Evading the control of its frontier developer or user.

  (2) “Catastrophic risk” does not include a foreseeable and material risk from any of the following:

    (A) Information that a foundation model outputs if the information is otherwise publicly accessible in a substantially similar form from a source other than a foundation model.

    (B) Lawful activity of the federal government.

    (C) Harm caused by a foundation model in combination with other software where the foundation model did not materially contribute to the harm.

(b) “Covered employee” means an employee responsible for assessing, managing, or addressing risk of critical safety incidents.

SB 53 only grants new whistleblower protections to a remarkably narrow group of people—namely, formal employees of frontier developers who have explicit responsibility for risk management. Independent contractors, unpaid advisors, board members, and staff of external evaluators don't count. Neither do researchers or engineers employed by a frontier developer whose job responsibilities don't include risk management. All of these people could plausibly have access to private information indicating a catastrophic risk, but SB 53 does nothing to strengthen their whistleblower rights.

(c) “Critical safety incident” means any of the following:

  (1) Unauthorized access to, modification of, or exfiltration of the model weights of a foundation model that results in death, bodily injury, or damage to, or loss of, property.

  (2) Harm resulting from the materialization of a catastrophic risk.

  (3) Loss of control of a foundation model causing death or bodily injury.

  (4) A foundation model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk.

(d) “Foundation model” has the meaning defined in Section 22757.11 of the Business and Professions Code.

(e) “Frontier developer” has the meaning defined in Section 22757.11 of the Business and Professions Code.

(f) “Large frontier developer” has the meaning defined in Section 22757.11 of the Business and Professions Code.

1107.1.

(a) A frontier developer shall not make, adopt, enforce, or enter into a rule, regulation, policy, or contract that prevents a covered employee from disclosing, or retaliates against a covered employee for disclosing, information to the Attorney General, a federal authority, a person with authority over the covered employee, or another covered employee who has authority to investigate, discover, or correct the reported issue, if the covered employee has reasonable cause to believe that the information discloses either of the following:

  (1) The frontier developer’s activities pose a specific and substantial danger to the public health or safety resulting from a catastrophic risk.

This provision protects whistleblowers who come forward with evidence that a large frontier developer's activities pose a catastrophic risk. Before SB 53, California law only protected disclosures of evidence that an employer's activities break (federal, state, or local) law.

Many ways that a developer could cause a catastrophic risk would also involve breaking the law, but a developer could in principle do something catastrophically dangerous yet legal. This provision could also give AI whistleblowers more confidence to exercise their rights, since it may be easier for a would-be whistleblowers to tell whether their employer is causing a catastrophic risk than to tell whether their employer is breaking a specific law.

  (2) The frontier developer has violated Chapter 25.1 (commencing with Section 22757.10) of Division 8 of the Business and Professions Code.

(b) A frontier developer shall not enter into a contract that prevents a covered employee from making a disclosure protected under Section 1102.5.

§1102.5 of the California Labor Code says that an employer may not make a rule against employees disclosing violations of federal, state, or local law to public officials or to their superiors. Further, it forbids an employer from retaliating against an employee who makes such a disclosure.

(c) A covered employee may use the hotline described in Section 1102.7 to make reports described in subdivision (a).

(d) A frontier developer shall provide a clear notice to all covered employees of their rights and responsibilities under this section, including by doing either of the following:

Existing California law (§1102.8 of the Labor Code) already requires every employer in the state to post a notice somewhere in the workplace to inform employees of their whistleblower rights. Subsection 2 goes further, requiring a frontier developer to send every covered employee an annual notice of their rights. This second, stronger requirement does not apply to other employees under generic whistleblower law.

  (1) At all times posting and displaying within any workplace maintained by the frontier developer a notice to all covered employees of their rights under this section, ensuring that any new covered employee receives equivalent notice, and ensuring that any covered employee who works remotely periodically receives an equivalent notice.

  (2) At least once each year, providing written notice to each covered employee of the covered employee’s rights under this section and ensuring that the notice is received and acknowledged by all of those covered employees.

(e) (1) A large frontier developer shall provide a reasonable internal process through which a covered employee may anonymously disclose information to the large frontier developer if the covered employee believes in good faith that the information indicates that the large frontier developer’s activities present a specific and substantial danger to the public health or safety resulting from a catastrophic risk or that the large frontier developer violated Chapter 25.1 (commencing with Section 22757.10) of Division 8 of the Business and Professions Code, including a monthly update to the person who made the disclosure regarding the status of the large frontier developer’s investigation of the disclosure and the actions taken by the large frontier developer in response to the disclosure.

  (2) (A) Except as provided in subparagraph (B), the disclosures and responses of the process required by this subdivision shall be shared with officers and directors of the large frontier developer at least once each quarter.

    (B) If a covered employee has alleged wrongdoing by an officer or director of the large frontier developer in a disclosure or response, subparagraph (A) shall not apply with respect to that officer or director.

(f) The court is authorized to award reasonable attorney’s fees to a plaintiff who brings a successful action for a violation of this section.

(g) In a civil action brought pursuant to this section, once it has been demonstrated by a preponderance of the evidence that an activity proscribed by this section was a contributing factor in the alleged prohibited action against the covered employee, the frontier developer shall have the burden of proof to demonstrate by clear and convincing evidence that the alleged action would have occurred for legitimate, independent reasons even if the covered employee had not engaged in activities protected by this section.

(h) (1) In a civil action or administrative proceeding brought pursuant to this section, a covered employee may petition the superior court in any county wherein the violation in question is alleged to have occurred, or wherein the person resides or transacts business, for appropriate temporary or preliminary injunctive relief.

  (2) Upon the filing of the petition for injunctive relief, the petitioner shall cause notice thereof to be served upon the person, and thereupon the court shall have jurisdiction to grant temporary injunctive relief as the court deems just and proper.

  (3) In addition to any harm resulting directly from a violation of this section, the court shall consider the chilling effect on other covered employees asserting their rights under this section in determining whether temporary injunctive relief is just and proper.

  (4) Appropriate injunctive relief shall be issued on a showing that reasonable cause exists to believe a violation has occurred.

  (5) An order authorizing temporary injunctive relief shall remain in effect until an administrative or judicial determination or citation has been issued, or until the completion of a review pursuant to subdivision (b) of Section 98.74, whichever is longer, or at a certain time set by the court. Thereafter, a preliminary or permanent injunction may be issued if it is shown to be just and proper. Any temporary injunctive relief shall not prohibit a frontier developer from disciplining or terminating a covered employee for conduct that is unrelated to the claim of the retaliation.

(i) Notwithstanding Section 916 of the Code of Civil Procedure, injunctive relief granted pursuant to this section shall not be stayed pending appeal.

(j) (1) This section does not impair or limit the applicability of Section 1102.5, including with respect to the rights of employees who are not covered employees to report violations of this chapter or Chapter 25.1 (commencing with Section 22757.10) of Division 8 of the Business and Professions Code.

  (2) The remedies provided by this section are cumulative to each other and the remedies or penalties available under all other laws of this state.

1107.2.

The loss of value of equity does not count as damage to or loss of property for the purposes of this chapter.

SECTION 5.

(a) The provisions of this act are severable. If any provision of this act or its application is held invalid, that invalidity shall not affect other provisions or applications that can be given effect without the invalid provision or application.

(b) This act shall be liberally construed to effectuate its purposes.

(c) The duties and obligations imposed by this act are cumulative with any other duties or obligations imposed under other law and shall not be construed to relieve any party from any duties or obligations imposed under other law and do not limit any rights or remedies under existing law.

(d) This act shall not apply to the extent that it strictly conflicts with the terms of a contract between a federal government entity and a frontier developer.

(e) This act shall not apply to the extent that it is preempted by federal law.

The federal government came close to passing such a preemption in summer 2025. A draft of the One Big Beautiful Bill Act passed by the House would have banned state and local governments from "regulating AI models, systems, or automated decision systems” for ten years. The Senate removed this preemption clause from the BBB in a last-minute vote, but Congress is still considering preempting state AI regulations.

(f) This act preempts any rule, regulation, code, ordinance, or other law adopted by a city, county, city and county, municipality, or local agency on or after January 1, 2025, specifically related to the regulation of frontier developers with respect to their management of catastrophic risk.

SECTION 6.

The Legislature finds and declares that Section 2 of this act, which adds Chapter 25.1 (commencing with Section 22757.10) to Division 8 of the Business and Professions Code, imposes a limitation on the public’s right of access to the meetings of public bodies or the writings of public officials and agencies within the meaning of Section 3 of Article I of the California Constitution. Pursuant to that constitutional provision, the Legislature makes the following findings to demonstrate the interest protected by this limitation and the need for protecting that interest:

Information in critical safety incident reports, assessments of risks from internal use, and reports from covered employees may contain information that could threaten public safety or compromise the response to an incident if disclosed to the public.

© 2025 Miles Kodama.
Content: All rights reserved.
Design & Code: Licensed under MIT License.