Unfortunately, I doubt that aligning the AIs to serve humans, not to treat them as a different race which is not to be destroyed, is even solvable. Even the ‘slowdown ending’ contains a brilliant line: “Humanity could easily become a society of superconsumers, spending our lives in an opium haze of amazing AI-provided luxuries and entertainment.” How can we trust the superhuman AIs to respect the human parasites if most human-done revolutions are closely related to the fact that the ruling classes were not as competent as humans hoped? And if the ruling class is hopelessly more stupid than the aligned AIs? At the very best, the AIs might just try to leave mankind alone...
Unfortunately, I doubt that aligning the AIs to serve humans, not to treat them as a different race which is not to be destroyed, is even solvable. Even the ‘slowdown ending’ contains a brilliant line: “Humanity could easily become a society of superconsumers, spending our lives in an opium haze of amazing AI-provided luxuries and entertainment.” How can we trust the superhuman AIs to respect the human parasites if most human-done revolutions are closely related to the fact that the ruling classes were not as competent as humans hoped? And if the ruling class is hopelessly more stupid than the aligned AIs? At the very best, the AIs might just try to leave mankind alone...