I do buy this, but note this requires fairly drastic actions that essentially amount to a pivotal act using an AI to coup society and government, because they have a limited time window in which to act before economic incentives means that most of the others kill/enslave almost everyone else.
Contra cousin_it, I basically don’t buy the story that power corrupts/changes your values, instead it corrupts your world model because there’s a very large incentive for your underlings to misreport things that flatter you, but conditional on technical alignment being solved, this doesn’t matter anymore, so I think power-grabs might not result in as bad of an outcome as we feared.
But this does require pretty massive changes to ensure the altruists stay in power, and they are not prepared to think about what this will mean.
I do buy this, but note this requires fairly drastic actions that essentially amount to a pivotal act using an AI to coup society and government, because they have a limited time window in which to act before economic incentives means that most of the others kill/enslave almost everyone else.
Contra cousin_it, I basically don’t buy the story that power corrupts/changes your values, instead it corrupts your world model because there’s a very large incentive for your underlings to misreport things that flatter you, but conditional on technical alignment being solved, this doesn’t matter anymore, so I think power-grabs might not result in as bad of an outcome as we feared.
But this does require pretty massive changes to ensure the altruists stay in power, and they are not prepared to think about what this will mean.