Sure: so EY arguing for a multinational agreement to just stop AGI altogether and airstrike data centres isn’t that radical at all, it’s actually a formalization of that equilibrium which might both keep the peace and save the world. If there is no creating UFAI without destroying the world, and there is no creating FAI without starting WW3, options are limited.
(that said, I don’t think a “pivotal act” requires what we would call world conquest in every possible world. In a world in which UFAI can indeed kill everyone via nanotech, similarly FAI can disseminate guardian nanomachines ready to short-circuit any nascent UFAI if the need arises and do absolutely nothing else. That would be a pretty peaceful and unobtrusive pivotal act)
Yes, so even if you creating friendly AI other will try to airstrike you. I wrote about it here Military AI as a Convergent Goal of Self-Improving AI.
Sure: so EY arguing for a multinational agreement to just stop AGI altogether and airstrike data centres isn’t that radical at all, it’s actually a formalization of that equilibrium which might both keep the peace and save the world. If there is no creating UFAI without destroying the world, and there is no creating FAI without starting WW3, options are limited.
(that said, I don’t think a “pivotal act” requires what we would call world conquest in every possible world. In a world in which UFAI can indeed kill everyone via nanotech, similarly FAI can disseminate guardian nanomachines ready to short-circuit any nascent UFAI if the need arises and do absolutely nothing else. That would be a pretty peaceful and unobtrusive pivotal act)