I think essays like this are not very helpful for the AI safety agenda. In fact, they seem quite likely to do more harm than good?
I see dozens of arguments for and against specific AI risk models every day. A large fraction of these arguments (especially in the Twitter and podcast circles where people think deeply about AI and often work in AI risk or frontier labs) are against the Yudkowsky positions in IABIED. These are often arguments made by very smart people, well versed in the literature (including Yudkowsky’s writing), who have significant meta-cognitive awareness when it comes to their own intellectual weaknesses.
All this is to say: of all these “live” debates that I see play out every day, none involve the type of IQ/intelligence skepticism that is criticized in this argument. People debate the effects on power-seeking of differing RL task lengths, the speed of super-exponential AI-assisted research takeoff, and the emergence of AI situational awareness with regard to testing environments in frontier models, but no one that I can see, in any significant position of power, is falling for the farcical reasoning employed by Mr. Humman. Even Simon Lermen’s example of “misuse” of comparative advantage (https://www.lesswrong.com/posts/tBr4AtpPmwhgfG4Mw/comparative-advantage-and-ai) is a fine educational example for the specific labor dynamics of the period we’re entering now, where AI use and human labor coexist. In any event other than the (unlikely) event of a ASI takeoff in hours, there will be a period of years if not decades where comparative advantage holds for human and AI labor. Is there anyone who’s arguing from Ricardo about the likelihood of AI risk? Maybe Marc Andresseen types? IQ is even more lopsided — who in silicon valley in the year of our lord 2025 is an IQ denier? Is the Trump admin somehow unfamiliar with the concept that people or machines can be very smart?
Anyways, all this is to say: who is this post for? Why write and promote arguments against irrelevant incorrect positions instead of addressing the massive substantive issues where intelligent, informed people disagree? Why take a continual “outside view” of the AI risk debate instead of grappling with the substantive issues that sway the opinions of informed insiders at labs and in politics? Will smart critics of AI safety arguments resent that their side is represented in Yudkowsky posts by a cretin with the reflective acumen of a kindergartener?
I think essays like this are not very helpful for the AI safety agenda. In fact, they seem quite likely to do more harm than good?
I see dozens of arguments for and against specific AI risk models every day. A large fraction of these arguments (especially in the Twitter and podcast circles where people think deeply about AI and often work in AI risk or frontier labs) are against the Yudkowsky positions in IABIED. These are often arguments made by very smart people, well versed in the literature (including Yudkowsky’s writing), who have significant meta-cognitive awareness when it comes to their own intellectual weaknesses.
All this is to say: of all these “live” debates that I see play out every day, none involve the type of IQ/intelligence skepticism that is criticized in this argument. People debate the effects on power-seeking of differing RL task lengths, the speed of super-exponential AI-assisted research takeoff, and the emergence of AI situational awareness with regard to testing environments in frontier models, but no one that I can see, in any significant position of power, is falling for the farcical reasoning employed by Mr. Humman. Even Simon Lermen’s example of “misuse” of comparative advantage (https://www.lesswrong.com/posts/tBr4AtpPmwhgfG4Mw/comparative-advantage-and-ai) is a fine educational example for the specific labor dynamics of the period we’re entering now, where AI use and human labor coexist. In any event other than the (unlikely) event of a ASI takeoff in hours, there will be a period of years if not decades where comparative advantage holds for human and AI labor. Is there anyone who’s arguing from Ricardo about the likelihood of AI risk? Maybe Marc Andresseen types? IQ is even more lopsided — who in silicon valley in the year of our lord 2025 is an IQ denier? Is the Trump admin somehow unfamiliar with the concept that people or machines can be very smart?
Anyways, all this is to say: who is this post for? Why write and promote arguments against irrelevant incorrect positions instead of addressing the massive substantive issues where intelligent, informed people disagree? Why take a continual “outside view” of the AI risk debate instead of grappling with the substantive issues that sway the opinions of informed insiders at labs and in politics? Will smart critics of AI safety arguments resent that their side is represented in Yudkowsky posts by a cretin with the reflective acumen of a kindergartener?