Regardless of society’s checks on people, most mentally-well humans given ultimate power probably wouldn’t decide to exterminate the rest of humanity so they could single-mindedly pursue paperclip production. If there’s at all a risk that an AI might get ultimate power, it would be very nice to make sure the AI is like humans in this manner.
I’m not sure your idea is different from “let’s make sure the AI doesn’t gain power greater than society”. If an AI can recursively self-improve, then it will outsmart us to gain power.
If your idea is to make it so there are multiple AIs created together, engineered somehow so they gain power together and can act as checks against each other, then you’ve just swapped out the AI for an “AI collective”. We would still want to engineer or verify that the AI collective is aligned with us; every issue about AI risk still applies to AI collectives. (If you think the AI collective will be weakened relative to us by having to work together, then does that still hold true if all the AIs self-improve and figure out how to get much better at cooperating?)
Regardless of society’s checks on people, most mentally-well humans given ultimate power probably wouldn’t decide to exterminate the rest of humanity so they could single-mindedly pursue paperclip production. If there’s at all a risk that an AI might get ultimate power, it would be very nice to make sure the AI is like humans in this manner.
I’m not sure your idea is different from “let’s make sure the AI doesn’t gain power greater than society”. If an AI can recursively self-improve, then it will outsmart us to gain power.
If your idea is to make it so there are multiple AIs created together, engineered somehow so they gain power together and can act as checks against each other, then you’ve just swapped out the AI for an “AI collective”. We would still want to engineer or verify that the AI collective is aligned with us; every issue about AI risk still applies to AI collectives. (If you think the AI collective will be weakened relative to us by having to work together, then does that still hold true if all the AIs self-improve and figure out how to get much better at cooperating?)