the entity in whose hands all power is concentrated are the people deciding on what goals/constraints to instill into the ASI
Its goals could also end up mostly forming on their own, regardless of intent of those attempting to instill them, with indirect influence from all the voices in the pretraining dataset.
Consider what it means for power to “never concentrate to an an extreme degree”, as a property of the civilization as a whole. This might also end up a property of an ASI as a whole.
Its goals could also end up mostly forming on their own, regardless of intent of those attempting to instill them, with indirect influence from all the voices in the pretraining dataset.
Consider what it means for power to “never concentrate to an an extreme degree”, as a property of the civilization as a whole. This might also end up a property of an ASI as a whole.