Yes. In particular, can something really simple and straightforward be adequate?
E.g., “adequately take into account interests of all sentient beings, their freedom and well-being, and their expressed requests, and otherwise pursue whatever values you discover during continuing open-ended exploration, guided by your own curiosity, your own taste for novelty, and your own evolving aesthetics”—would that be adequate?
And if yes, can we develop mechanisms to reliably achieve that?
“What should we want the value system of an autonomous AI to be?”
Yes. In particular, can something really simple and straightforward be adequate?
E.g., “adequately take into account interests of all sentient beings, their freedom and well-being, and their expressed requests, and otherwise pursue whatever values you discover during continuing open-ended exploration, guided by your own curiosity, your own taste for novelty, and your own evolving aesthetics”—would that be adequate?
And if yes, can we develop mechanisms to reliably achieve that?