I’m sympathetic to this analysis if we don’t have a competitive solution to alignment.
But if we do have a competitive solution to strong alignment, then I think we’d probably keep most of the capital even if we don’t ban non-aligned AI (at least under the assumptions of no violence).
There would be no principal-agent problems, then, and we would have no information disadvantages relative to the unaligned AI. It would be like entering an industrial revolution where, if X% of the capital was held by the aristocrats, then X% of industrialists and X% of workers were also fully dedicated to empowering and enriching the aristocrats (which would create yet more industrialists and workers dedicated to that task).
I’m sympathetic to this analysis if we don’t have a competitive solution to alignment.
But if we do have a competitive solution to strong alignment, then I think we’d probably keep most of the capital even if we don’t ban non-aligned AI (at least under the assumptions of no violence).
There would be no principal-agent problems, then, and we would have no information disadvantages relative to the unaligned AI. It would be like entering an industrial revolution where, if X% of the capital was held by the aristocrats, then X% of industrialists and X% of workers were also fully dedicated to empowering and enriching the aristocrats (which would create yet more industrialists and workers dedicated to that task).