My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.
My issue here is with “past experience”. We don’t have past experience of developing AGI. If this was about secular cycles in agricultural societies where boundary conditions remain the same over millennia, I’d be much more sympathetic. But lack of past experience is inherent to new technologies. Inferring future technological progress from the past necessitates shaky analogies. You can see any pattern you want and deduce any conclusion you want from history, by cherry-picking the technology, the time-window and the metric. You say “Wright Brothers proved experts are Luddites”, I say ” Where is the flying car I’ve been promised”. There is no way to not cherry-pick. Zoom in far enough and any curve looks smooth, including a hard AI takeoff.
My point is don’t look at Wright Brothers, the Manhattan Project or Moore’s Law, look at streamlines, atomic mass spectra and the Landauer limit to infer where we’re headed. Even if the picture is incomplete it’s still more informative than vague analogies with the past.
What does streamlines refer to in this context? And what is the relevance of atomic mass spectra?
Looking at atomic mass spectra of uranium and its fission products (and hence the difference in their energy potential) in the early 20th century would have helped you predict just how big a deal nuclear weapons will be, in a way that looking at the rate of improvement of conventional explosives would not have.