Dumb question: In the omnicide outcome agi continues its existence, but whether it’s sentient or not no one else from earth does. In the undignified survival outcome agi continues its existence and potentially so do many humans for functionally ever and those human may have lives better than death (even barely). Why is parochial value-alignment necessarily worse than no alignment at all?
I think there’s a possibility that their lives, or some of them, are vastly worse than death. See the recent post the case against value alignment for some pretty convincing concerns.
“Undignified survival” likely also involves many people/entities whose lives are worse than death. E. g., being eternally tortured because they committed some crime that the parochial values consider unforgivable, or because these entities aren’t recognized as morally relevant by the parochial values and are therefore exploited for work/entertainment.
Dumb question: In the omnicide outcome agi continues its existence, but whether it’s sentient or not no one else from earth does. In the undignified survival outcome agi continues its existence and potentially so do many humans for functionally ever and those human may have lives better than death (even barely). Why is parochial value-alignment necessarily worse than no alignment at all?
I think there’s a possibility that their lives, or some of them, are vastly worse than death. See the recent post the case against value alignment for some pretty convincing concerns.
“Undignified survival” likely also involves many people/entities whose lives are worse than death. E. g., being eternally tortured because they committed some crime that the parochial values consider unforgivable, or because these entities aren’t recognized as morally relevant by the parochial values and are therefore exploited for work/entertainment.