In particular: at present, frontier AI algorithms improve quite fast. Ho et al (2024) estimate that “the level of compute needed to achieve a given level of performance has halved roughly every 8 months, with a 95% confidence interval of 5 to 14 months.”
A mistake people often make when computing algorithmic progress is to include the effects of algorithmic improvements that didn’t cause much step-change in efficiency but instead improved the future slope of the scaling curve, and to then count the resulting ongoing increase in hardware improvements from the changed slope as steady algorithmic progress even though without the hardware improvements whose effect the boosted they wouldn’t have happened. Both of the algorithmic improvements analyzed in the source you link to are of exactly this form: both the development of transformer and the Chinchilla scaling laws were scaling slope changes far more than step chnages. However, if hardware stops improving, as would presumably be the case during a monitored red-lit pause, then these improvements would also stop: it doesn’t matter what the slope of the scaling curve is if you’re not currently scaling your hardware. So the interesting question, for this purpose, is what the rate of purely step-change algorithmic improvements would be if hardware is static. There have been improvements of this type, quite a few of them, but they also tend to be individually quite modest: 10% here, 20% there. Unfortunately the people who make these estimates often omit to produce this, more useful, number. But that’s the number that’s needed for both assessing pause feasibility, and for looking at the risk of a software-only intelligence explosion.
I suggest you look for good current estimates of those numbers, and update your post to link to them. My understanding is that they’re significantly less scary.
A mistake people often make when computing algorithmic progress is to include the effects of algorithmic improvements that didn’t cause much step-change in efficiency but instead improved the future slope of the scaling curve, and to then count the resulting ongoing increase in hardware improvements from the changed slope as steady algorithmic progress even though without the hardware improvements whose effect the boosted they wouldn’t have happened. Both of the algorithmic improvements analyzed in the source you link to are of exactly this form: both the development of transformer and the Chinchilla scaling laws were scaling slope changes far more than step chnages. However, if hardware stops improving, as would presumably be the case during a monitored red-lit pause, then these improvements would also stop: it doesn’t matter what the slope of the scaling curve is if you’re not currently scaling your hardware. So the interesting question, for this purpose, is what the rate of purely step-change algorithmic improvements would be if hardware is static. There have been improvements of this type, quite a few of them, but they also tend to be individually quite modest: 10% here, 20% there. Unfortunately the people who make these estimates often omit to produce this, more useful, number. But that’s the number that’s needed for both assessing pause feasibility, and for looking at the risk of a software-only intelligence explosion.
I suggest you look for good current estimates of those numbers, and update your post to link to them. My understanding is that they’re significantly less scary.