Will California AI bill (SB-1047) be signed into law with the "kill switch" requirement?
19
Ṁ15k
Oct 2
2%
chance

The "SB-1047 Safe and Secure Innovation for Frontier Artificial Intelligence Models Act" is scheduled to be voted on by the California General Assembly in August 2024, and if passed should be signed or vetoed by the governor before the end of September.

The bill contains a requirement for the publishers of large AI models to include a "kill switch": a capability to stop the model completely if it is deemed dangerous. This requirement can be interpreted as basically banning open source distribution of the models.

Will the bill be turned into law before October 1st 2024, including this specific requirement?

Similar more general question: /ZviMowshowitz/will-california-bill-sb-1047-become

Update from 2024-08-23: There has been some discussion in the comments regarding whether the original text of the bill actually blocked releasing model weights. In any case, I just want to confirm that if the bill doesn't require shutting down the derivative models that you don't control, I'll resolve as NO.

Get Ṁ1,000 play money
Sort by:
opened a Ṁ500 NO at 5% order

FYI because a lot of people are confused about this - there was never a kill switch requirement, models under the control of others were never part of the shutdown requirement. People just misread it. https://legiscan.com/CA/text/SB1047/id/2919384

“Full shutdown” [...] all copies and derivative models, on all computers and storage devices within the custody, control, or possession of a person

Discussion at https://thezvi.substack.com/p/q-and-a-on-proposed-sb-1047

(I'm just trying to clear up the confusion about the bill itself, this is not about market resolution which I think we are clear on.)

this has been changed to not apply to models under the control of others. How does that affect your likely resolution?

If it doesn't block e.g. releasing a model as open source then it's a NO. Basically if it just affects the entity that runs the model, but not the entity that releases the model, then it's a NO.

As I understand it, that requirement has already been removed, so this should resolve NO?

I'm not sure whether the text of the bill can be further changed, so I prefer to wait until the final version is signed.

opened a Ṁ500 NO at 5% order

FYI because a lot of people are confused about this - there was never a kill switch requirement, models under the control of others were never part of the shutdown requirement. People just misread it. https://legiscan.com/CA/text/SB1047/id/2919384

“Full shutdown” [...] all copies and derivative models, on all computers and storage devices within the custody, control, or possession of a person

Discussion at https://thezvi.substack.com/p/q-and-a-on-proposed-sb-1047

(I'm just trying to clear up the confusion about the bill itself, this is not about market resolution which I think we are clear on.)

Maybe I'm misunderstanding something, but the original text that you quoted sounds like it requires being able to shut down the derivative models. The amended text says "All covered model derivatives controlled by a developer" which sounds weaker.

If the current definition of "shutdown" stays in the final text of the bill when it is signed into law, then I'll resolve the market to NO. If the text is somehow reverted to a stronger requirement to be able to shutdown all derivative models, I'll resolve to YES.

The key part of the quote is "within the custody, control, or possession" (that's from the original bill)

It says basically the same thing as the amended version, I think the amended version just moves the definition around so it's easier to parse.

"... of a person"

It's not clear to me what person is meant here.

This bill would require a person that operates a computing cluster, as defined, to implement appropriate written policies and procedures to do certain things when a customer utilizes compute resources that would be sufficient to train a covered model

Before initiating training of a covered model that is not a derivative model that is not the subject of a positive safety determination, and until that covered model is the subject of a positive safety determination, the developer of that covered model shall do all of the following:

Implement the capability to promptly enact a full shutdown of the covered model.

“Full shutdown” means the cessation of operation of a covered model, including all copies and derivative models, on all computers and storage devices within custody, control, or possession of a person, including any computer or storage device remotely provided by agreement.

I believe "a person" here refers to the developer. They must be able to shut down models under their control.

The amended bill makes this clearer by pulling the language about control up from the definition of full shutdown. But as far as I can tell this is for ease of understanding, the definition looks the same.

Ok, maybe you are right. In any case, I just want to confirm that if the bill doesn't require shutting down the derivative models that you don't control, I'll resolve as NO.

Yeah, this is not about market resolution which I think we are clear on, I'm just trying to clear up the confusion about the bill itself.

I'm glad they amended this part of the bill to make it unambiguous.

Your title asks if the bill will “pass”. This language refers to a bill being approved by a legislative body (House, Senate, or Congress). Not necessarily being signed by the executive and enacted. Later in the description, you ask if it will become law. You may want to bring the title and description language into alignment.

Thank you for the correction. I updated the title.