By Anna Tong

SAN FRANCISCO (Reuters) – California lawmakers passed a hotly contested artificial-intelligence safety bill on Wednesday, which requires one more procedural vote before it goes to Governor Gavin Newsom, who has until Sept. 30 to sign or veto it.

Tech companies developing generative AI, capable of producing text, images, or audio in response to prompts, have largely opposed the legislation, known as SB 1047, arguing it may push AI firms out of California and stifle innovation.

Some Democrats in U.S. Congress, including Representative Nancy Pelosi, also voiced opposition. However, proponents like Tesla CEO Elon Musk, who runs an AI firm called xAI, support the bill.

The measure mandates safety testing for advanced AI models costing over $100 million to develop or requiring significant computing power. It also requires developers to establish methods for disabling AI models in case of malfunction, effectively a ‘kill switch.’

Additionally, the bill empowers the state attorney general to sue non-compliant developers, especially if there’s an ongoing threat, such as AI compromising government systems like the power grid.

Furthermore, the bill mandates that developers engage third-party auditors to evaluate their safety practices and offers protections to whistleblowers reporting AI misuse.

The bill’s author, Democratic State Senator Scott Wiener, who represents San Francisco, the home of OpenAI and many AI startups, states that legislation is essential to safeguard the public against potentially uncontrollable AI advances.

Martin Casado, a general partner at venture capital firm Andreessen Horowitz, expressed hope that Newsom would veto the bill, highlighting extensive bipartisan opposition.

Companies like Alphabet’s Google, Microsoft-backed OpenAI, and Meta Platforms raised concerns via letters to Wiener. Amazon-backed Anthropic mentioned that while the bill’s benefits could outweigh its costs, it still perceived some aspects as troubling or vague.