The main thing that seems wrong to me, similar to some of your other recent posts, is that AI progress seems to mysteriously decelerate around 2030. I predict that things will look much more sci-fi after that point than in your story (if we’re still alive).
The scenario does not say that AI progress slows down. What I imagined to be happening is that after 2028 or so, there is AI research being done by AIs at unprecedented speeds, and this drives raw intelligence forward more and more, but (1) the AIs still need to run expensive experiments to make progress sometimes, and (2) basically nothing is bottlenecked by raw intelligence anymore so you don’t really notice it getting even better.
Seems we have a big disagreement about the real-world effects of superintelligence, then. I agree they’ll be bottlenecked on a bunch of stuff, but when I try to estimate how fast things will be going overall (i.e. how much those bottlenecks will bite) I end up thinking something like a year or two till robotic economy comparable to grass, whereas you seem to be thinking doubling times will hover around 1 year for decades. I’d love to discuss sometime. tbc there’s a lot of uncertainty, I’m not confident, etc.
The big reason why such a slowdown could happen is that the hyper-fast scaling trends can’t last beyond 2030, which has been the main driver of AI progress, and I still expect it to be the main driver to 2030, and if there’s no real way for AI systems to get better past that point through algorithmic advances, then this story becomes much more plausible.
It’s more that it stops being relevant to humans, as keeping humans in the loop slows down the exponential growth
I do think VR and neuralink-like tech will be a very big deal though, especially in regards to allowing people experiences that would otherwise be expensive in atoms
Thanks I enjoyed this.
The main thing that seems wrong to me, similar to some of your other recent posts, is that AI progress seems to mysteriously decelerate around 2030. I predict that things will look much more sci-fi after that point than in your story (if we’re still alive).
The scenario does not say that AI progress slows down. What I imagined to be happening is that after 2028 or so, there is AI research being done by AIs at unprecedented speeds, and this drives raw intelligence forward more and more, but (1) the AIs still need to run expensive experiments to make progress sometimes, and (2) basically nothing is bottlenecked by raw intelligence anymore so you don’t really notice it getting even better.
Seems we have a big disagreement about the real-world effects of superintelligence, then. I agree they’ll be bottlenecked on a bunch of stuff, but when I try to estimate how fast things will be going overall (i.e. how much those bottlenecks will bite) I end up thinking something like a year or two till robotic economy comparable to grass, whereas you seem to be thinking doubling times will hover around 1 year for decades. I’d love to discuss sometime. tbc there’s a lot of uncertainty, I’m not confident, etc.
The big reason why such a slowdown could happen is that the hyper-fast scaling trends can’t last beyond 2030, which has been the main driver of AI progress, and I still expect it to be the main driver to 2030, and if there’s no real way for AI systems to get better past that point through algorithmic advances, then this story becomes much more plausible.
It’s more that it stops being relevant to humans, as keeping humans in the loop slows down the exponential growth
I do think VR and neuralink-like tech will be a very big deal though, especially in regards to allowing people experiences that would otherwise be expensive in atoms