I’m a little worried I don’t think the argument from algorithmic progress works as stated.
Let’s assume Agent-0 or 1 take us from today where AI tools generously can reduce labor hours for a researcher writing code full-time by 5-15% to a world where it can reduce a researcher’s total labor time by 80% (where checking code and tests visually, hitting confirmations, and skimming their own research papers as they get produced independently are the only tasks an AI researcher is doing). With that kind of best-case situation, doesn’t it still only affect OpenAI’s total velocity marginally, since all of their statements suggest the major input constraint is chips?
I think there’s a bit of a two-step going on:
Propose algorithmic progress as a separable firm-level intermediate product that lacks (or has minimal) chip constraints, instead mostly “AI Researcher” labor input.
Then we assert massive increases in labor productivity and/or quality of AI Researcher labor (based on industry-secret future agent improvements).
Then we get a virtuous cycle where the firm can reinvest the algorithmic progress back into labor productivity and quality increases.
I think there’s a few counterarguments:
Most of the increases in algorithmic progress in your own source came at high costs of compute (e.g. Chinchilla scaling laws), so it’s unlikely sustained algorithmic progress does not require a very large chip input.
If labor productivity and quality were about to massively increase, and labor input increases can enable a firm-level virtuous cycle while not being subject to chip constraints and there are large uncertainties around said productivity and quality increases, OpenAI would be increasing mass of labor at any cost.
OpenAI is not focused on hiring, they are focused on acquiring chips, even in the medium term, so they do not believe this flywheel will occur anytime soon.
Another big problem is that algorithmic progress is very much not firm-level, and attempts to monopolize algorithmic improvements are very hard to enforce even in the short-term (outside things that get very authoritarian and censorious fast)
Your estimate that algorithmic progress currently is half of capabilities progress is likely inflated by 3-10x, as the vast majority of percentage-wise algorithmic progress happens before the transformer is released, and post-2018 algorithmic progress estimated as a percentage of progress is about 5-15% (see below)
5. The doubling rate of algorithmic progress in terms of effective compute (via that paper) appears quite stable even as the amount of AI researcher labor dedicated to it has multiplied to a huge degree, so even if we do grant an immediate 5x increase in effective AI researcher labor at OpenAI, there’s little reason to expect it to do much beyond a one-time jump of a year’s purely algorithmic progress or so (or about ~1.5-8x effective compute or a couple months total progress) (given OpenAI fraction of labor pool etc. and with some lag for adjustment).
To achieve an ongoing 1.5x AI R&D progress multiplier (and this multiplier applies to the 5-15% of all progress not the 50% of all progress), you need:
The productivity gain for AI researchers to be enough for OpenAI’s AI Research labor to equal the rest of the world (give-or-take, but 5x productivity seems a reasonable lower estimate).
Then, through hiring and/or even further productivity/labor quality multiplier improvements, OpenAI needs to increase their labor input by a yearly factor approx. equaling the yearly growth rate in total AI Researcher labor since 2018, probably upwards of an additional 1.5-2x increase in effective AI Researcher labor per subsequent year ongoing forever.
This all is assuming there is zero marginal decline in input efficiency of AI Researcher labor, so if you need even a few percent more chips per labor unit the virtuous cycle burns out very fast.
The basic story is just that there is no plausible amount of non-breakthrough R&D that will make up for the coming slowing of compute doubling times. Attempts to hold on to monopolistic positions through hoarding research or chips will fail sooner than later because we are not getting capabilities explosion in the next 2-3 years (or 10, but ymmv).
Thanks for this! Lots to think about!
I’m a little worried I don’t think the argument from algorithmic progress works as stated.
Let’s assume Agent-0 or 1 take us from today where AI tools generously can reduce labor hours for a researcher writing code full-time by 5-15% to a world where it can reduce a researcher’s total labor time by 80% (where checking code and tests visually, hitting confirmations, and skimming their own research papers as they get produced independently are the only tasks an AI researcher is doing). With that kind of best-case situation, doesn’t it still only affect OpenAI’s total velocity marginally, since all of their statements suggest the major input constraint is chips?
I think there’s a bit of a two-step going on:
Propose algorithmic progress as a separable firm-level intermediate product that lacks (or has minimal) chip constraints, instead mostly “AI Researcher” labor input.
Then we assert massive increases in labor productivity and/or quality of AI Researcher labor (based on industry-secret future agent improvements).
Then we get a virtuous cycle where the firm can reinvest the algorithmic progress back into labor productivity and quality increases.
I think there’s a few counterarguments:
Most of the increases in algorithmic progress in your own source came at high costs of compute (e.g. Chinchilla scaling laws), so it’s unlikely sustained algorithmic progress does not require a very large chip input.
If labor productivity and quality were about to massively increase, and labor input increases can enable a firm-level virtuous cycle while not being subject to chip constraints and there are large uncertainties around said productivity and quality increases, OpenAI would be increasing mass of labor at any cost.
OpenAI is not focused on hiring, they are focused on acquiring chips, even in the medium term, so they do not believe this flywheel will occur anytime soon.
Another big problem is that algorithmic progress is very much not firm-level, and attempts to monopolize algorithmic improvements are very hard to enforce even in the short-term (outside things that get very authoritarian and censorious fast)
Your estimate that algorithmic progress currently is half of capabilities progress is likely inflated by 3-10x, as the vast majority of percentage-wise algorithmic progress happens before the transformer is released, and post-2018 algorithmic progress estimated as a percentage of progress is about 5-15% (see below)
5. The doubling rate of algorithmic progress in terms of effective compute (via that paper) appears quite stable even as the amount of AI researcher labor dedicated to it has multiplied to a huge degree, so even if we do grant an immediate 5x increase in effective AI researcher labor at OpenAI, there’s little reason to expect it to do much beyond a one-time jump of a year’s purely algorithmic progress or so (or about ~1.5-8x effective compute or a couple months total progress) (given OpenAI fraction of labor pool etc. and with some lag for adjustment).
To achieve an ongoing 1.5x AI R&D progress multiplier (and this multiplier applies to the 5-15% of all progress not the 50% of all progress), you need:
The productivity gain for AI researchers to be enough for OpenAI’s AI Research labor to equal the rest of the world (give-or-take, but 5x productivity seems a reasonable lower estimate).
Then, through hiring and/or even further productivity/labor quality multiplier improvements, OpenAI needs to increase their labor input by a yearly factor approx. equaling the yearly growth rate in total AI Researcher labor since 2018, probably upwards of an additional 1.5-2x increase in effective AI Researcher labor per subsequent year ongoing forever.
This all is assuming there is zero marginal decline in input efficiency of AI Researcher labor, so if you need even a few percent more chips per labor unit the virtuous cycle burns out very fast.
The basic story is just that there is no plausible amount of non-breakthrough R&D that will make up for the coming slowing of compute doubling times. Attempts to hold on to monopolistic positions through hoarding research or chips will fail sooner than later because we are not getting capabilities explosion in the next 2-3 years (or 10, but ymmv).