Ok; what do you think of Soares’s claim that SB-1047 should have been made stronger and the connections to existential risk should have been made more clearly? That seems probably false to me.
I think that if it were to go ahead, it should have been made stronger and clearer. But this wouldn’t have been politically feasible, and therefore if that were the standard being aimed for it wouldn’t have gone ahead.
This I think would have been better than the outcome that actually happened.
All I know is that a lot of organizations seem to be shy talking about the AI takeover risk, and the endorsements the book got surprised me, regarding how receptive government officials are. (Considering how little cherry-picking they did.)
My very uneducated guess is that Newsom vetoed the bill because he was more of a consequentialist/Longtermist than the cause-driven lawmakers who passed the bill, so one can argue the failure mode was a “lack of appeal to consequentialist interests.” One might argue “it passed by cause-driven lawmakers by a wide margin, but got blocked by the consequentialist.” But the cause-driven vs. consequentialist motives are pure speculation, I know nothing about these people aside from Newsom’s explanation...
From my perspective, FWIW, the endorsements we got would have been surprising even if they had been maximally cherry-picked. You usually just can’t find cherries like those.
Ok; what do you think of Soares’s claim that SB-1047 should have been made stronger and the connections to existential risk should have been made more clearly? That seems probably false to me.
I think that if it were to go ahead, it should have been made stronger and clearer. But this wouldn’t have been politically feasible, and therefore if that were the standard being aimed for it wouldn’t have gone ahead.
This I think would have been better than the outcome that actually happened.
To be honest, I don’t know.
All I know is that a lot of organizations seem to be shy talking about the AI takeover risk, and the endorsements the book got surprised me, regarding how receptive government officials are. (Considering how little cherry-picking they did.)
My very uneducated guess is that Newsom vetoed the bill because he was more of a consequentialist/Longtermist than the cause-driven lawmakers who passed the bill, so one can argue the failure mode was a “lack of appeal to consequentialist interests.” One might argue “it passed by cause-driven lawmakers by a wide margin, but got blocked by the consequentialist.” But the cause-driven vs. consequentialist motives are pure speculation, I know nothing about these people aside from Newsom’s explanation...
From my perspective, FWIW, the endorsements we got would have been surprising even if they had been maximally cherry-picked. You usually just can’t find cherries like those.