The only lesson to learn from history is to not learn lessons from it, especially when something as freaky and unprecedented as AGI is concerned.
This seems like a pretty wild claim to me, even as someone who agrees that AGI is freaky and unprecedented, possibly to the point that we should expect it to depart drastically from past experience.
My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.
This seems like a pretty wild claim to me, even as someone who agrees that AGI is freaky and unprecedented, possibly to the point that we should expect it to depart drastically from past experience.
My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
What does streamlines refer to in this context? And what is the relevance of atomic mass spectra?
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.