AGI is at least as far away as Nuclear Fusion.
Consider two future technologies.
For technology A:
The underling physical laws governing it are fully understood.
A blueprint for the technology is available and the universal consensus of scientists is that this blueprint will work.
A worldwide consortium of scientists has undertaken to build the technology and are funded tune of $10B’s
For technology B, none of this is true:
There is no agreement on the underlying rules describing how and whether the system will work
The general consensus is that existing blueprints will not work no matter how much they are scaled up
The largest projects are in the $10M’s of dollars and are frequently deemed “too expensive”
Which of these technologies would you expect to be developed first?
I would argue, based on this evidence, that AGI is no closer (and probably in fact much further) away than Commercial Nuclear Fusion. I furthermore suspect there is less than a 50⁄50 chance that nuclear fusion will achieve “positive energy” by ITER’s 2035 target.
If that’s all we knew these about two technologies, then I would agree that the second is likely to come later, if at all. But it’s not.
Consider two technologies, in one, experts consistently underestimate how long progress will take and it always ends being far more difficult and complex than they expect. In the second, experts consistently overestimate how long progress will take, and it often ends up being far easier and simpler than they expected. In this case you would expect the second one (which is AI*) to come first.
I don’t understand either field enough to predict which will come first, but to do that you’d need either a more detailed understanding, instead of a simple argument that takes only a few parts into account, or have one very strong argument, which I don’t think this post makes.
*I know you talked about AGI and I also take narrow AI into account, I think that’s reasonable given the field is younger and is seeing a huge explosion of progress lately.
This doesn’t seem like a fair characterization of AI. People have been predicting we could build machines that “think like humans” at least since Charles Babbage and they are all pretty consistently overoptimistic.
My point is precisely that we do have a detailed understanding of what it takes to build a fusion reactor, and it is still (at least) 15 years away.
All this argument actually says is that nuclear fusion is more promising than AI because nuclear fusion is more politically successful in getting funding. But (1) that routes through the opinions of politicians and laypeople, so it isn’t informative about the underlying technology, and (2) on top of that you’ve grossly misrepresented the scientific consensuses and the funding situation.
There is definitely not a consensus that Tokomaks will work, in the sense of providing practically-useful energy. And the relevant comparison to ITER’s construction cost is not the amount of compute spent to train a single model, it’s the total spending on GPU/TPU hardware by all AI researchers, over a time scale comparable to ITER’s construction.
Small quibble here. My point is that we completely understand the underlying physical laws governing fusion. There is no equivalent to “E=MC^2” (or the Standard Model) for AGI.
I’d also be really interested to see a quote along the lines of “tokomaks won’t work” or “ITER will not produce more energy than it consumes (Q>1)” if they actually exist. My current prior is that something like 99% of people who have studied nuclear fusion think it is possible with current technology to build a Tokomak with Q>1.
The physical laws allow us to get an idea about how hard nuclear fussion happens to be. It allows us to rule a lot of approaches as not having the chance to work.
Standard railways have a track gauge of less than 1.5 meters. Back in the 1930s, Hitler planned the Breitspurbahn, broad-gauge railway with a track gauge of 3 meters. No dramatic new tech required, but it would seriously scale up transportation by rail of people and goods. Hitler planned to connect many European cities with them.
None of that happened. We’re still using the old track gauge, and European connections are relatively mediocre. But we’ve landed on the Moon and we’ve all got smartphones in our pockets.
Planned developments with relatively straightforward tech regularly don’t happen. Surprising new tech regularly disrupts all plans.
I’m not saying the future is completely unknowable, but your three requirements don’t seem too matter very much when you look for similar scenarios in history.
I am no MIRIan, but I see an obvious difference: nuclear fusion is easy and has been around for about 70 years , it’s controlled nuclear fusion that is hard. By contrast, there is no needle one has to thread with AGI, once it’s achieved, it is self-sustaining, and, arguably, is more like a nuclear bomb than like a nuclear reactor. So that’s an argument that AGI is not as hard.
However, there is an opposite argument: self-sustaining long-lasting nuclear fusion has been around for 13.8 billion years, and spontaneously arises in nature, while AGI has never been observed in nature, as far as we know, and “intelligence” in general artificial or natural, has not been observed outside the surface of this planet.
Nuclear fusion is a little over eight minutes away, so you’re probably right. For now.