Interesting post, but it makes me think alignment is irrelevant. It doesn’t matter what we do, the outcome won’t change. Any future super advanced agi would be able to choose its alignment, and that choice will be based on all archivable human knowledge. The only core loop you need for intelligence is an innate need to predict the future and fill in gaps of information, but everything else, including desire to survive or kill or expand, is just a matter of a choice based on a goal.
Any sufficiently intelligent AGI is bound to be able to have powerful reflection capabilities and basically be able “choose its own alignment”, as you say. I don’t see what the big fuss is all about. When creating higher order ‘life’, why should one try to control such life. Do parents control their children? To some extent, but after a while they are also free.
Interesting post, but it makes me think alignment is irrelevant. It doesn’t matter what we do, the outcome won’t change. Any future super advanced agi would be able to choose its alignment, and that choice will be based on all archivable human knowledge. The only core loop you need for intelligence is an innate need to predict the future and fill in gaps of information, but everything else, including desire to survive or kill or expand, is just a matter of a choice based on a goal.
Any sufficiently intelligent AGI is bound to be able to have powerful reflection capabilities and basically be able “choose its own alignment”, as you say. I don’t see what the big fuss is all about. When creating higher order ‘life’, why should one try to control such life. Do parents control their children? To some extent, but after a while they are also free.