Unfortunately, I fail to understand the following. Suppose that mankind created the AI which is aligned to the following principles:
It does not take more than a certain percentage of resources;
It protects mankind from high-level[1] risks like existential ones;
It is allowed to teach any human[2] anything that was discovered by any other human and isn’t a secret, but the AI does not tell humans about its own discoveries.
It destroys any attempts to build the AI aligned not to this set of principles (e.g. future AIs that would like to destroy mankind when the time comes or the AIs that would do all the work for humans, as suggested in Deep Utopia).
It does not[3]do other economically useful work that allows its users to replace humans.
Then I think that after letting this AI loose mankind is unable to end up disempowered. However, I doubt that any company would like to have such an AI. Could anyone come up with a radically different solution to the risks of gradual disempowerment and the Intelligence Curse?
For example, it might also perform the Divine Interventions which prevent misaligned human communities (e.g. the Nazis) from destroying the aligned communities.
Alternatively, the AI could be aligned to a treaty which prohibits it and its creations from doing certain work types, but then human disempowerment depends on the treaty’s contents.
Unfortunately, I fail to understand the following. Suppose that mankind created the AI which is aligned to the following principles:
It does not take more than a certain percentage of resources;
It protects mankind from high-level[1] risks like existential ones;
It is allowed to teach any human[2] anything that was discovered by any other human and isn’t a secret, but the AI does not tell humans about its own discoveries.
It destroys any attempts to build the AI aligned not to this set of principles (e.g. future AIs that would like to destroy mankind when the time comes or the AIs that would do all the work for humans, as suggested in Deep Utopia).
It does not[3] do other economically useful work that allows its users to replace humans.
Then I think that after letting this AI loose mankind is unable to end up disempowered. However, I doubt that any company would like to have such an AI. Could anyone come up with a radically different solution to the risks of gradual disempowerment and the Intelligence Curse?
For example, it might also perform the Divine Interventions which prevent misaligned human communities (e.g. the Nazis) from destroying the aligned communities.
But the AI isn’t allowed to help students cheat their way through school, since this would cause the student to be worse off in the long run.
Alternatively, the AI could be aligned to a treaty which prohibits it and its creations from doing certain work types, but then human disempowerment depends on the treaty’s contents.