But I sometimes see people attribute the impressive and scary suddenness of deployable nuclear weapons entirely to the physics of energy release from a supercritical mass, and I think this is a mistake.
I disagree. I think it is a mistake to shoehorn “patterns” onto the history of technological progress where you deliberately pick the time window and the metric and ignore timescale in order to fit a narrative.
I don’t know what motivates people to try to dissolve historical discontinuities such as the advent of the nuclear bomb, but they did manage to find a metric along which early nuclear bombs were comparable to conventional bombs, namely explosive yield per dollar. But the real importance of the atom bomb is that it’s possible at all; that physics allows it; that it is about a million times more energy-dense than chemical explosives—not a hundred, not a trillion; a million. That is what determined the post-ww2 strategic landscape and the predicament humanity is currently in. That which is determined by the laws of nature and not the dynamics of human societies.
You can’t get that information out of drawing lines through the rate of improvement of explosive yields or whatever. You wouldn’t even have thought of drawing that particular line. This whole exercise is mistaking hindsight for wisdom. The only lesson to learn from history is to not learn lessons from it, especially when something as freaky and unprecedented as AGI is concerned.
The only lesson to learn from history is to not learn lessons from it, especially when something as freaky and unprecedented as AGI is concerned.
This seems like a pretty wild claim to me, even as someone who agrees that AGI is freaky and unprecedented, possibly to the point that we should expect it to depart drastically from past experience.
My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.
I disagree. I think it is a mistake to shoehorn “patterns” onto the history of technological progress where you deliberately pick the time window and the metric and ignore timescale in order to fit a narrative.
I don’t know what motivates people to try to dissolve historical discontinuities such as the advent of the nuclear bomb, but they did manage to find a metric along which early nuclear bombs were comparable to conventional bombs, namely explosive yield per dollar. But the real importance of the atom bomb is that it’s possible at all; that physics allows it; that it is about a million times more energy-dense than chemical explosives—not a hundred, not a trillion; a million. That is what determined the post-ww2 strategic landscape and the predicament humanity is currently in. That which is determined by the laws of nature and not the dynamics of human societies.
You can’t get that information out of drawing lines through the rate of improvement of explosive yields or whatever. You wouldn’t even have thought of drawing that particular line. This whole exercise is mistaking hindsight for wisdom. The only lesson to learn from history is to not learn lessons from it, especially when something as freaky and unprecedented as AGI is concerned.
This seems like a pretty wild claim to me, even as someone who agrees that AGI is freaky and unprecedented, possibly to the point that we should expect it to depart drastically from past experience.
My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
What does streamlines refer to in this context? And what is the relevance of atomic mass spectra?
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.