Well, this is the saddest I’ve been since April 1st 2022.
It really sucks that SB 1047 didn’t pass. I don’t know if Anthropic could have gotten it passed if they had said “dudes this this fucking important, pass it now” instead of “for some reason we should wait until things are already
It is nice that at least Anthropic did still get to show up to the table, and that they said anything at all. I sure wish their implied worldview didn’t seem so crazy. (I really don’t get how you can think it’s workable to race here, even if you think Phase I alignment is easy, as well as it seeming really wrong to think Phase I alignment is that likely to be easy)
It feels like winning pathways right now mostly route through:
Some kind of miracle of Vibe Shift (ideally mediated through a miracle of Sanity). I think this needs masterwork-level communication / clarity / narrative setting.
Just… idk, somehow figure out how to just Solve The Hard Part Real Fast.
Somehow muddle through with scary demos that get a few key people to change their mind before it’s too late.
It’s possible that we won’t get something that deserves the name ASI or TAI until, for example, 2030. And a lot can change in more than 5 years!
The current panic seems excessive. We do not live in a world where all reasonable people expect the emergence of artificial superintelligence in the next few years and the extinction of humanity soon after that. The situation is very worrying, and this is the most likely cause of death for all of us in the coming years, yes. But I don’t understand how anyone can be so sure of a bad outcome as to consider people’s survival a miracle.
Well, this is the saddest I’ve been since April 1st 2022.
It really sucks that SB 1047 didn’t pass. I don’t know if Anthropic could have gotten it passed if they had said “dudes this this fucking important, pass it now” instead of “for some reason we should wait until things are already
It is nice that at least Anthropic did still get to show up to the table, and that they said anything at all. I sure wish their implied worldview didn’t seem so crazy. (I really don’t get how you can think it’s workable to race here, even if you think Phase I alignment is easy, as well as it seeming really wrong to think Phase I alignment is that likely to be easy)
It feels like winning pathways right now mostly route through:
Some kind of miracle of Vibe Shift (ideally mediated through a miracle of Sanity). I think this needs masterwork-level communication / clarity / narrative setting.
Just… idk, somehow figure out how to just Solve The Hard Part Real Fast.
Somehow muddle through with scary demos that get a few key people to change their mind before it’s too late.
It’s possible that we won’t get something that deserves the name ASI or TAI until, for example, 2030.
And a lot can change in more than 5 years!
The current panic seems excessive. We do not live in a world where all reasonable people expect the emergence of artificial superintelligence in the next few years and the extinction of humanity soon after that.
The situation is very worrying, and this is the most likely cause of death for all of us in the coming years, yes. But I don’t understand how anyone can be so sure of a bad outcome as to consider people’s survival a miracle.
It seems that a Vibe Shift is possible—if Trump’s voters would realize what OpenAI and others are actually trying to build, they would be furious.
As far as why Anthropic should probably race, here’s @joshc’s take on it, using the fictional company Magma as an example:
https://www.lesswrong.com/posts/8vgi3fBWPFDLBBcAx/planning-for-extreme-ai-risks#5__Heuristic__1__Scale_aggressively_until_meaningful_AI_software_R_D_acceleration
The other winning pathways I can list are:
1. Unlearning becomes more effective, such that you can use AI control strategies much easier.
2. We are truly in an alignment is easy world, where giving it data mostly straightforwardly changes it’s values.
3. We somehow muddle through, with an outcome that none of us expected.