Serial speed advantage of AI is sufficient to deliver an ancient AI civilization within a few years of the first AGI even without superintelligence or significant algorithmic self-improvement, simply by AIs thinking faster than humans, and using this advantage to quickly develop hardware that allows them to think even faster. No need to posit exponential growth when just the first two steps let you start running past the competition at 10,000 times their speed.
How much of the work to create better hardware can be done in a computer’s head, though? I have no doubt that smarter being can create better hardware than we have now, but are there other real world limitations that would very quickly limit the rate of improvement. I imagine even something much smarter than us would need to experiment in the physical world, as well as build new machines (and mine the necessary materials) and do a lot of actual physical work that would take time and that computers can not obviously do 10,000x faster than humans.
In this scenario AGIs are unrestricted in their activity, so in particular they can do physical experiments if that turns out to be useful. Manufacturing of improved hardware at scale requires development of physical tools anyway, so it’s a step along the way.
The starting point is likely biotechnology, with enough mastery to design novel organisms. Think drosophila swarms, not whales, but with an option to assemble into whales. With enough energy, this gives both exponential scaling and control at small scale, which is good for a large number of diverse experiments that run very quickly. Macroscale biotechnology replaces human physical infrastructure, both crude manipulation of matter (logistics, assembly) and chemical processing. More subtle objects like chips and fusion plants could be manufactured with specialized non-biological machines, the same way humans do such things, but backed by the exponential biological infrastructure and enough planning to get everything working right away. If diamondoid nanotech turns out to be feasible, this works even better, but it doesn’t have to.
(Of course, this is all under the absolutely impossible assumption of lack of superintelligence. So exploratory engineering, not prediction. A lower bound on what seems feasible, even if it never becomes worthwhile to do in a remotely similar way.)
Serial speed advantage of AI is sufficient to deliver an ancient AI civilization within a few years of the first AGI even without superintelligence or significant algorithmic self-improvement, simply by AIs thinking faster than humans, and using this advantage to quickly develop hardware that allows them to think even faster. No need to posit exponential growth when just the first two steps let you start running past the competition at 10,000 times their speed.
How much of the work to create better hardware can be done in a computer’s head, though? I have no doubt that smarter being can create better hardware than we have now, but are there other real world limitations that would very quickly limit the rate of improvement. I imagine even something much smarter than us would need to experiment in the physical world, as well as build new machines (and mine the necessary materials) and do a lot of actual physical work that would take time and that computers can not obviously do 10,000x faster than humans.
In this scenario AGIs are unrestricted in their activity, so in particular they can do physical experiments if that turns out to be useful. Manufacturing of improved hardware at scale requires development of physical tools anyway, so it’s a step along the way.
The starting point is likely biotechnology, with enough mastery to design novel organisms. Think drosophila swarms, not whales, but with an option to assemble into whales. With enough energy, this gives both exponential scaling and control at small scale, which is good for a large number of diverse experiments that run very quickly. Macroscale biotechnology replaces human physical infrastructure, both crude manipulation of matter (logistics, assembly) and chemical processing. More subtle objects like chips and fusion plants could be manufactured with specialized non-biological machines, the same way humans do such things, but backed by the exponential biological infrastructure and enough planning to get everything working right away. If diamondoid nanotech turns out to be feasible, this works even better, but it doesn’t have to.
(Of course, this is all under the absolutely impossible assumption of lack of superintelligence. So exploratory engineering, not prediction. A lower bound on what seems feasible, even if it never becomes worthwhile to do in a remotely similar way.)