Thanks for this important work. One thing I remain uncertain about is whether the scenario’s rapid physical and economic scaling rests on multiple unstated (mutually dependent) assumptions:
The AI infrastructure boom must avoid a major investment correction before AI systems generate enough real economic value to justify the expenditure. If returns arrive later than expected, overcapacity, defaults, or a broader financial shock could sharply reduce capex and produce something resembling an AI funding winter.
Compute expansion must not be bottlenecked for years by grid connections, power generation, HBM, advanced packaging, fab capacity, or fab equipment. You model some of these constraints, but I am not convinced that the physical buildout can proceed on the implied timescale.
Robotics must become capable of reliably constructing and operating power plants, fabs, mines, and robot factories without depending on unpredictable breakthroughs in embodied autonomy: long-horizon reliability, dexterity, continual adaptation, and generalization to novel real-world situations. This seems partly bootstrapping-dependent, because rapid robotic infrastructure growth is then used to justify the compute growth enabling further AI progress.
How sensitive is the scenario to these assumptions? In particular, what happens if there is a multi-year capex contraction, an energy or semiconductor plateau, or a substantial lag between cognitive automation and broadly capable robotics?
These things point me to worrying more about the economy than these loss of control scenarios. Maybe you can correct me on that.
Indeed, moral philosophy is the main bottleneck for AI progress.
Two thoughts:
I’ve come to the conclusion that there can be no “complete” system of morality – only a method akin to the scientific method. Just as claims about nature can be disproven or reinterpreted, moral claims must always remain open to revision in light of changing circumstances. I’m currently working on such a method.
Even a perfectly rational moral system would fail to align AI with people fully, because humans can be deeply irrational. Therefore, no form of moral perfection can be relied upon in principle.