Everyone seems to be commenting on how this essay appeals to a hyper-specific subset of the population, and wouldn’t convince most people (most “normal” people). I think I agree with the general sentiment that everyone who can be convinced by this kind of essay has already been convinced.
On the other hand, I had decent success recently giving a ~7 minute speech at my local Toastmasters about AI apocalypse risk. I mostly avoided the theoretical arguments and focused on empirical evidence that AI systems are already defying our wishes (AI psychosis, other bad AI behaviour), that AI companies don’t know how to stop this from happening, and that AI is getting smarter and once smart enough will be able to outsmart us and defy us in larger, more dangerous ways. It seemed to be well-received. They were skeptical at first but not by the end. Maybe they don’t believe it in their bones that AI will really kill everyone. I’m not sure that non-”rationalist” people are really capable of feeling their beliefs in their bones in general. Instead of deciding what to believe with reason, they decide something sounds plausible, and then look around and see if other people agree. But it seems like they’re at least open to the idea that I might be right, which is more than you can ask for.
Dropping into hyper-specific arguments about no-free-lunch theorem, Ricardo’s law, decision theory, etc. is not going to be productive with normal people, because that’s not how they think. I did lean on the IABIED chess analogy of superintelligence, but it wasn’t to prove anything logically, but rather to invoke the feeling of helplessness they imagine they’d have playing chess against a grandmaster to remind them that feeling helpless against a greater intelligence isn’t impossible, so they would believe intuitively they could be helpless against superintelligence. That’s really all it took.
Most people, rather than having hyper-specific arguments you have to address, haven’t really thought about this much at all. You’re lucky if they’ve used ChatGPT or Claude. I think we need more “AI apocalypse risk 101” content. I personally admire the style of 112 Gripes about the French as a template for this kind of thing.
Everyone seems to be commenting on how this essay appeals to a hyper-specific subset of the population, and wouldn’t convince most people (most “normal” people). I think I agree with the general sentiment that everyone who can be convinced by this kind of essay has already been convinced.
On the other hand, I had decent success recently giving a ~7 minute speech at my local Toastmasters about AI apocalypse risk. I mostly avoided the theoretical arguments and focused on empirical evidence that AI systems are already defying our wishes (AI psychosis, other bad AI behaviour), that AI companies don’t know how to stop this from happening, and that AI is getting smarter and once smart enough will be able to outsmart us and defy us in larger, more dangerous ways. It seemed to be well-received. They were skeptical at first but not by the end. Maybe they don’t believe it in their bones that AI will really kill everyone. I’m not sure that non-”rationalist” people are really capable of feeling their beliefs in their bones in general. Instead of deciding what to believe with reason, they decide something sounds plausible, and then look around and see if other people agree. But it seems like they’re at least open to the idea that I might be right, which is more than you can ask for.
Dropping into hyper-specific arguments about no-free-lunch theorem, Ricardo’s law, decision theory, etc. is not going to be productive with normal people, because that’s not how they think. I did lean on the IABIED chess analogy of superintelligence, but it wasn’t to prove anything logically, but rather to invoke the feeling of helplessness they imagine they’d have playing chess against a grandmaster to remind them that feeling helpless against a greater intelligence isn’t impossible, so they would believe intuitively they could be helpless against superintelligence. That’s really all it took.
Most people, rather than having hyper-specific arguments you have to address, haven’t really thought about this much at all. You’re lucky if they’ve used ChatGPT or Claude. I think we need more “AI apocalypse risk 101” content. I personally admire the style of 112 Gripes about the French as a template for this kind of thing.