Meditations on Moloch is an excellent piece—but it’s not the argument I’m making.
Scott describes how competition leads to suboptimal outcomes, yes. But he stops at describing the problem. He doesn’t draw the specific conclusion that AGI alignment is structurally impossible because any attempt to slow down or “align” will be outcompeted by systems that don’t bother. He also doesn’t apply that conclusion to the AGI race with the same blunt finality I do: this ends in extinction, and it cannot be stopped.
So unless you can point to the section where Scott actually follows the AGI race dynamics to the conclusion that alignment will be systematically optimised away—rather than just made “more difficult”—then no, that essay doesn’t make my argument. It covers part of the background context. That’s not the same thing.
This kind of reply—“here’s a famous link that kind of gestures in the direction of what you’re talking about”—is exactly the vague dismissal I’ve been calling out. If my argument really has been made before, someone should be able to point to where it’s clearly laid out.
So far, no one has. The sidestepping and lack of direct engagement in my arguments in this comment section alone has to be studied.
https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
It’s “mainstream” here, described well many times before.
Meditations on Moloch is an excellent piece—but it’s not the argument I’m making.
Scott describes how competition leads to suboptimal outcomes, yes. But he stops at describing the problem. He doesn’t draw the specific conclusion that AGI alignment is structurally impossible because any attempt to slow down or “align” will be outcompeted by systems that don’t bother. He also doesn’t apply that conclusion to the AGI race with the same blunt finality I do: this ends in extinction, and it cannot be stopped.
So unless you can point to the section where Scott actually follows the AGI race dynamics to the conclusion that alignment will be systematically optimised away—rather than just made “more difficult”—then no, that essay doesn’t make my argument. It covers part of the background context. That’s not the same thing.
This kind of reply—“here’s a famous link that kind of gestures in the direction of what you’re talking about”—is exactly the vague dismissal I’ve been calling out. If my argument really has been made before, someone should be able to point to where it’s clearly laid out.
So far, no one has. The sidestepping and lack of direct engagement in my arguments in this comment section alone has to be studied.