Other possible success stories are semi-success stories, where the outcome is not very good, but some humans survive and significant part of human values is preserved.
One case of the semi-success story is that many sovereign AIs control different countries or territories and implement different values in them. In some of these territories AIs’ values will be very close to the best possible implementation of aligned AI. Other AIs could be completely inhuman. Slow takeoff could end in such world of many AIs.
Another case is that unfriendly AI decides not to kill humans for some instrumental reasons (research, acausal trade with other AI, just not bother killing them). It could even run many simulations of human history including simulations of friendly AIs-sovereigns and their human civilizations. In that case, many people will live very happy lives despite being controlled by unfriendly AI. Like some people were happy under Saddam Hussein rule.
Semi-success stories could be seen as a more natural outcome as we typically don’t have perfect things in life.
Other possible success stories are semi-success stories, where the outcome is not very good, but some humans survive and significant part of human values is preserved.
One case of the semi-success story is that many sovereign AIs control different countries or territories and implement different values in them. In some of these territories AIs’ values will be very close to the best possible implementation of aligned AI. Other AIs could be completely inhuman. Slow takeoff could end in such world of many AIs.
Another case is that unfriendly AI decides not to kill humans for some instrumental reasons (research, acausal trade with other AI, just not bother killing them). It could even run many simulations of human history including simulations of friendly AIs-sovereigns and their human civilizations. In that case, many people will live very happy lives despite being controlled by unfriendly AI. Like some people were happy under Saddam Hussein rule.
Semi-success stories could be seen as a more natural outcome as we typically don’t have perfect things in life.