What if Japan had surrendered before the weapons could be deployed, and the Manhattan project had never been completed? I could totally believe in a one in one hundred thousand probability that nuclear weapons just never saw proliferation, maybe more
Specifically, I am referring to after the Trinity test. They had assembled a device and released about 25 kilotons. I am claiming for AI, the “Trinity test” already happened—llms, game playing RL agents that beat humans, and all the other 2022 AI results show that larger yields are possible. Trinity had “no direct significance” on the world in that it didn’t blow up a city, and the weapon wasn’t deployable on a missile, but it showed both were readily achievable (“make a reactor and produce plutonium or separate out u235”) and fission yield was real.
In our world, we don’t have AI better than humans at everything, and they aren’t yet affecting real world products much, but I would argue that the RL agent results show that the equivalent to “fission yield”, “superintelligence”, is possible and also readily achievable. (big neural network, big compute, big training data set = superintelligence)
After the Trinity test, if Japan had surrendered, the information had already leaked, and motivated agents—the all the world powers—would have started power seeking to make nukes. What sort of plausible history has them not doing it? A worldwide agreement that they won’t? What happens when, in a nuke free world, one country attacks another. Won’t the target start rush developing fission weapons? Won’t the other powers learn of this and join in?
It’s unstable. A world where all the powers agree not to build nukes is not a stable one, any pertubation will push it to history states closer to our real timeline.
I would argue that such a world with agreements not to build AGI is similarly not stable.
What if Japan had surrendered before the weapons could be deployed, and the Manhattan project had never been completed? I could totally believe in a one in one hundred thousand probability that nuclear weapons just never saw proliferation, maybe more
Specifically, I am referring to after the Trinity test. They had assembled a device and released about 25 kilotons. I am claiming for AI, the “Trinity test” already happened—llms, game playing RL agents that beat humans, and all the other 2022 AI results show that larger yields are possible. Trinity had “no direct significance” on the world in that it didn’t blow up a city, and the weapon wasn’t deployable on a missile, but it showed both were readily achievable (“make a reactor and produce plutonium or separate out u235”) and fission yield was real.
In our world, we don’t have AI better than humans at everything, and they aren’t yet affecting real world products much, but I would argue that the RL agent results show that the equivalent to “fission yield”, “superintelligence”, is possible and also readily achievable. (big neural network, big compute, big training data set = superintelligence)
After the Trinity test, if Japan had surrendered, the information had already leaked, and motivated agents—the all the world powers—would have started power seeking to make nukes. What sort of plausible history has them not doing it? A worldwide agreement that they won’t? What happens when, in a nuke free world, one country attacks another. Won’t the target start rush developing fission weapons? Won’t the other powers learn of this and join in?
It’s unstable. A world where all the powers agree not to build nukes is not a stable one, any pertubation will push it to history states closer to our real timeline.
I would argue that such a world with agreements not to build AGI is similarly not stable.