I think there is a missing step between the core argument of the article (no new physics remain to allow superpowerful bombs) and the layman conclusion (don’t be afraid of superweapons).
The reason bombs are terrifying is that they destroy value; traditional bombs do so by releasing a lot of energy in a chaotic manner, which disperses whatever I care about; other kinds of which you discuss homogenize everything into uninteresting states.
But it’s easy to make superweapons that don’t rely on new physics, at that level of abstraction: you only need something that can increase its own power, like engineered plagues, nanobots or memes. Even without a clear catalytic reaction on an elegantly simple level, they run on a substrate that is very expressive, allowing them to run extremely complex self-replication which more than compensates for the need to run on a complex substrate.
Saying that there is “no superweapon between the nuclear bomb and false vacuum decay” is a huge conceptual leap, and I think your conclusion could be much sharper (even if the disambiguation section on asteroids or lasers already pulls a lot of weight), like “New superweapons won’t come from theoretical physics research.”.
People may be afraid of strangelets or black holes because they think “oh, physicists discovered nuclear energy then made nukes, the same could happen with the LHC” and this article cleanly dissolves that narrow misconception, but “oh, scientists discovered powerful new technology then made nukes, the same could happen with AI” is very close, and is not invalidated by your article. This is why I think it’d be a mistake to learn a heuristic of not being afraid of big science stuff, even if just learning about all of this was fascinating.
I’ve edited the conclusion to try to make the scope clearer (I meant “between” in the physicist’s sense of phenomena at intermediate energy scales, so that chemical systems are explicitly out of scope; I realize now that’s not obvious). I certainly don’t believe that all big science stuff is definitely safe!
I think there is a missing step between the core argument of the article (no new physics remain to allow superpowerful bombs) and the layman conclusion (don’t be afraid of superweapons).
The reason bombs are terrifying is that they destroy value; traditional bombs do so by releasing a lot of energy in a chaotic manner, which disperses whatever I care about; other kinds of which you discuss homogenize everything into uninteresting states.
But it’s easy to make superweapons that don’t rely on new physics, at that level of abstraction: you only need something that can increase its own power, like engineered plagues, nanobots or memes. Even without a clear catalytic reaction on an elegantly simple level, they run on a substrate that is very expressive, allowing them to run extremely complex self-replication which more than compensates for the need to run on a complex substrate.
Saying that there is “no superweapon between the nuclear bomb and false vacuum decay” is a huge conceptual leap, and I think your conclusion could be much sharper (even if the disambiguation section on asteroids or lasers already pulls a lot of weight), like “New superweapons won’t come from theoretical physics research.”.
People may be afraid of strangelets or black holes because they think “oh, physicists discovered nuclear energy then made nukes, the same could happen with the LHC” and this article cleanly dissolves that narrow misconception, but “oh, scientists discovered powerful new technology then made nukes, the same could happen with AI” is very close, and is not invalidated by your article.
This is why I think it’d be a mistake to learn a heuristic of not being afraid of big science stuff, even if just learning about all of this was fascinating.
I’ve edited the conclusion to try to make the scope clearer (I meant “between” in the physicist’s sense of phenomena at intermediate energy scales, so that chemical systems are explicitly out of scope; I realize now that’s not obvious). I certainly don’t believe that all big science stuff is definitely safe!