Anonymous: I’m curious if the grim outlook is currently mainly due to technical difficulties or social/coordination difficulties. (Both avenues might have solutions, but maybe one seems more recalcitrant than the other?)
Eliezer Yudkowsky: Technical difficulties. Even if the social situation were vastly improved, on my read of things, everybody still dies because there is nothing that a handful of socially coordinated projects can do, or even a handful of major governments who aren’t willing to start nuclear wars over things, to prevent somebody else from building AGI and killing everyone 3 months or 2 years later. There’s no obvious winnable position into which to play the board.
Anonymous: just to clarify, that sounds like a large scale coordination difficulty to me (i.e., we—as all of humanity—can’t coordinate to not build that AGI).
Eliezer Yudkowsky: I wasn’t really considering the counterfactual where humanity had a collective telepathic hivemind? I mean, I’ve written fiction about a world coordinated enough that they managed to shut down all progress in their computing industry and only manufacture powerful computers in a single worldwide hidden base, but Earth was never going to go down that route. Relative to remotely plausible levels of future coordination, we have a technical problem.
I had a similar reflection in this Jan 2025 review of the Shut it All Down letter. In that thread I referenced Discussion with Eliezer Yudkowsky on AGI interventions, where the pause advocate was anonymous:
Good job anonymous questioner of November 2021. But that’s not much earlier than the 2022 sources you found, and it’s just asking questions. The other pause advocacy I found at the time was 2022 era: What an Actually Pessimistic Containment Strategy Looks Like in April 2022 and Let’s Think About Slowing Down AI in December 2022.