It’s weird when someone says “this tech I’m making has a 25% chance of killing everyone” and doesn’t add “the world would be better-off if everyone, including me, was stopped.”
It’s weird when someone says “I think my complicated idea for preventing destruction of the Earth has some chance of working” and doesn’t add “but it’d be crazy to gamble civilization on that.”
It’s weird when AI people look inward at me and say “overconfident” rather than looking outward at the world to say “Finally, a chance to speak! It is true, we should not be doing this. I have more hope than he does, but it’s far too dangerous. Better for us all to be stopped.”
You can say that without even stopping! It’s not even hypocritical, if you think you have a better chance than the next guy and the next guy is plowing ahead regardless.
It’s a noteworthy omission, when people who think they’re locked in a suicide race aren’t begging the world to stop it.
Yes, we have plenty of disagreements about the chance that the complex plans succeed. But it seems we all agree that the status quo is insane. Don’t forget to say that part too.
Say it loudly and clearly and often, if you believe it. It’s perhaps the most important thing to say.
The thing I want to draw attention to here is noticing the asymmetry in who you feel moved to complain about. The ubiquity of this phenonemon is why I think “normalcy bias” is (so far anyway) worse than “sensationalism bias.”
noticing the asymmetry in who you feel moved to complain about.
I think I basically complain when I see opinions that feel importantly wrong to me?
When I’m in very LessWrong-shaped spaces, that often looks like arguing in favor of “really shitty low-dignity approaches to getting the AIs to do our homework for us are >>1% to turn out okay, I think there’s lots of mileage in getting slightly less incompetent at the current trajectory”, and I don’t really harp on the “would be nice if everyone just stopped” thing the same way I don’t harp on the “2+2=4” thing, except to do virtue signaling to my interlocutor about not being an e/acc so I don’t get dismissed as being in the Bad Tribe Outgroup.
When I’m in spaces with people who just think working on AI is cool, I’m arguing about the “holy shit this is an insane dangerous technology and you are not oriented to it with anything like a reasonable amount of caution” thing, and I don’t really harp on the “some chance we make it out okay” bit except to signal that I’m not a 99.999% doomer so I don’t get dismissed as being in the Bad Tribe Outgroup.
I think the asymmetry complaint is very reasonable for writing that is aimed at a broad audience, TBC, but when people are writing LessWrong posts I think it’s basically fine to take the shared points of agreement for granted and spend most of your words on the points of divergence. (Though I do think it’s good practice to signpost that agreement at least a little.)
I wanna copy in a recent Nate tweet:
The thing I want to draw attention to here is noticing the asymmetry in who you feel moved to complain about. The ubiquity of this phenonemon is why I think “normalcy bias” is (so far anyway) worse than “sensationalism bias.”
A reply pretty near the top that also feels relevant to this overall point:
I think I basically complain when I see opinions that feel importantly wrong to me?
When I’m in very LessWrong-shaped spaces, that often looks like arguing in favor of “really shitty low-dignity approaches to getting the AIs to do our homework for us are >>1% to turn out okay, I think there’s lots of mileage in getting slightly less incompetent at the current trajectory”, and I don’t really harp on the “would be nice if everyone just stopped” thing the same way I don’t harp on the “2+2=4” thing, except to do virtue signaling to my interlocutor about not being an e/acc so I don’t get dismissed as being in the Bad Tribe Outgroup.
When I’m in spaces with people who just think working on AI is cool, I’m arguing about the “holy shit this is an insane dangerous technology and you are not oriented to it with anything like a reasonable amount of caution” thing, and I don’t really harp on the “some chance we make it out okay” bit except to signal that I’m not a 99.999% doomer so I don’t get dismissed as being in the Bad Tribe Outgroup.
I think the asymmetry complaint is very reasonable for writing that is aimed at a broad audience, TBC, but when people are writing LessWrong posts I think it’s basically fine to take the shared points of agreement for granted and spend most of your words on the points of divergence. (Though I do think it’s good practice to signpost that agreement at least a little.)
nod, fwiw I didn’t have this complaint about you.