Competitive agents will not choose to in order to beat the competition
Competitive agents will chose to commit suicide, knowing it’s suicide, to beat the competition? That suggests that we should observe CEOs mass-poisoning their employees, Jonestown-style, in a galaxy-brained attempt to maximize shareholder value. How come that doesn’t happen?
Are you quite sure the underlying issue here is not that the competitive agents don’t believe the suicide race to be a suicide race?
This is a mischaracterisation of the argument. I’m not saying competitive agents knowingly choose extinction. I’m saying the structure of the race incentivises behaviour that leads to extinction, even if no one intends it.
CEOs aren’t mass-poisoning their employees because that would damage their short and long-term competitiveness. But racing to build AGI—cutting corners on alignment, accelerating deployment, offloading responsibility—improves short-term competitiveness, even if it leads to long-term catastrophe. That’s the difference.
And what makes this worse is that even the AGI safety field refuses to frame it in those terms. They don’t call it suicide. They call it difficult. They treat alignment like a hard puzzle to be solved—not a structurally impossible task under competitive pressure.
So yes, I agree with your last sentence. The agents don’t believe it’s a suicide race. But that doesn’t counter my point—it proves it. We’re heading toward extinction not because we want to die, but because the system rewards speed over caution, power over wisdom. And the people who know best still can’t bring themselves to say it plainly.
This is exactly the kind of sleight-of-hand rebuttal that keeps people from engaging with the actual structure of the argument. You’ve reframed it into something absurd, knocked down the strawman, and accidentally reaffirmed the core idea in the process.
Competitive agents will chose to commit suicide, knowing it’s suicide, to beat the competition? That suggests that we should observe CEOs mass-poisoning their employees, Jonestown-style, in a galaxy-brained attempt to maximize shareholder value. How come that doesn’t happen?
Are you quite sure the underlying issue here is not that the competitive agents don’t believe the suicide race to be a suicide race?
This is a mischaracterisation of the argument. I’m not saying competitive agents knowingly choose extinction. I’m saying the structure of the race incentivises behaviour that leads to extinction, even if no one intends it.
CEOs aren’t mass-poisoning their employees because that would damage their short and long-term competitiveness. But racing to build AGI—cutting corners on alignment, accelerating deployment, offloading responsibility—improves short-term competitiveness, even if it leads to long-term catastrophe. That’s the difference.
And what makes this worse is that even the AGI safety field refuses to frame it in those terms. They don’t call it suicide. They call it difficult. They treat alignment like a hard puzzle to be solved—not a structurally impossible task under competitive pressure.
So yes, I agree with your last sentence. The agents don’t believe it’s a suicide race. But that doesn’t counter my point—it proves it. We’re heading toward extinction not because we want to die, but because the system rewards speed over caution, power over wisdom. And the people who know best still can’t bring themselves to say it plainly.
This is exactly the kind of sleight-of-hand rebuttal that keeps people from engaging with the actual structure of the argument. You’ve reframed it into something absurd, knocked down the strawman, and accidentally reaffirmed the core idea in the process.