A new bill in California aiming to regulate large frontier AI models has sparked significant resistance from various tech industry stakeholders, including startup founders, investors, AI researchers, and organizations advocating for open-source software. The bill, SB 1047, was introduced by California State Senator Scott Weiner.

Senator Weiner asserts that the bill mandates developers of large and powerful AI systems to adhere to common-sense safety standards. However, critics of the legislation argue that it would stifle innovation and jeopardize the entire AI industry.

In May, the California legislature passed the controversial bill, which is currently progressing through various committees. Following a final vote in August, the bill could be presented to Governor Gavin Newsom for signing into law. If enacted, SB 1047 would become the first major law in the United States regulating AI in a state that hosts many major tech companies.

Provisions of the Bill

Known as the 'Safe and Secure Innovation for Frontier Artificial Intelligence Systems Act,' SB 1047 aims to hold top AI companies like Meta, OpenAI, Anthropic, and Mistral accountable for the potential catastrophic dangers associated with rapidly advancing technology.

The bill specifically targets entities deploying large frontier AI models, defining "large" as those AI systems trained using computing power of 10^26 floating operations per second (FLOPS) with the training process costing more than $100 million. AI models fine-tuned using computing power greater than three times 10^25 FLOPS also fall under the bill's purview.

"If not properly subject to human controls, future development in artificial intelligence may also have the potential to be used to create novel threats to public safety and security, including by enabling the creation and the proliferation of weapons of mass destruction, such as biological, chemical, and nuclear weapons, as well as weapons with cyber-offensive capabilities," the bill states.

Liability and Compliance

According to the latest draft, developers behind large frontier AI models can be held liable for "critical harms," including using AI to create chemical or nuclear weapons and launching cyberattacks on critical infrastructure. It also covers human crimes committed by AI models with limited human oversight resulting in death, bodily injury, and property damage.

However, developers cannot be held responsible if AI-generated output leading to death or injury is information available elsewhere. The bill also mandates that AI models have an inbuilt kill switch for emergencies and prohibits the launch of large frontier AI models posing a reasonable risk of causing or enabling critical harm.

To ensure compliance, AI models must undergo independent audits by third-party auditors. Developers violating the bill's provisions could face legal action by California’s attorney general and must adhere to safety standards recommended by a new AI certifying body, the 'Frontier Model Division,' envisioned by the California government.

Controversy and Criticism

The draft legislation echoes concerns voiced by AI critics, including notable figures like Geoffrey Hinton and Yoshua Bengio, who believe AI could pose existential threats to humanity and thus requires regulation. The bill is also supported by the Centre for AI Safety, which published an open letter likening AI risks to those posed by nuclear wars or pandemics.

Despite this support, the bill has faced heavy criticism from many quarters. A major argument against it is that it could effectively eliminate open-source AI models. Open-source models ensure transparency and improved security as their inner workings can be freely accessed or modified by anyone. The proposed California bill, however, might discourage companies like Meta from making their AI models open source due to potential liability for misuse by other developers.

Conclusion

As California moves closer to potentially enacting SB 1047, the debate over regulating large frontier AI models continues to intensify. Proponents argue that the bill is necessary to safeguard against catastrophic harms posed by advanced AI technologies, while opponents fear it could stifle innovation and the open-source movement that has driven much of AI's progress.

The final outcome of this legislative effort will not only impact the tech industry in California but could also set a precedent for how AI is regulated across the United States and possibly the world. As the tech community grapples with these challenges, the balance between innovation and safety remains at the forefront of this critical discussion.