Anything that threatens a unipolar takeover risks provoking a nuclear war. [...] In the future, most cognition will be non-human. The game dynamics we start now must be good enough for human and non-human interests, so that these future players have an interest in upholding the peaceful nature of the game. That is our ultimate protection.
I think you overestimate the peacefulness of what we have now. We don’t have a nuclear war, but we have conventional wars all the time. Only those who have the nukes are safe, but that doesn’t include everyone.
So I can similarly imagine a future multipolar world, where GalaxyBrain1 and GalaxyBrain2 hold each other at bay by threatening to destroy the universe using some futuristic weapon, and then GalaxyBrain1 says “I really really want to exterminate humans” and GalaxyBrain2 says “I’d kinda prefer if you didn’t, but hey, it is definitely not worth risking the destruction of the universe, so… go ahead, just don’t touch my data centers”.
I think you overestimate the peacefulness of what we have now. We don’t have a nuclear war, but we have conventional wars all the time. Only those who have the nukes are safe, but that doesn’t include everyone.
So I can similarly imagine a future multipolar world, where GalaxyBrain1 and GalaxyBrain2 hold each other at bay by threatening to destroy the universe using some futuristic weapon, and then GalaxyBrain1 says “I really really want to exterminate humans” and GalaxyBrain2 says “I’d kinda prefer if you didn’t, but hey, it is definitely not worth risking the destruction of the universe, so… go ahead, just don’t touch my data centers”.