So, I am absolutely on-board with the GDP Growth & Measurement critique, and the Taiwan Supply Chain problem.
For the GDP Growth & Measurement critique
I think you make good points here, but there are additional reason to expect even more slowdowns. For instance, I don’t think it’s quite as easy as the report makes it out to be to ramp up automation. I suspect that ramping up of novel technologies, like new sorts of robotics, will lead to novel bottlenecks in supply chains. Things which previously weren’t scarce will become (temporarily) scarce. There is a significant amount of inertia in ramping up novel verticals.
Another reason is that there’s going to be a lot of pushback from industry that will create delays. Investment into existing infrastructure creates strong incentives from wealthy powerful actors to lobby for not having their recently-built infrastructure obsoleted. This political force, plus the political force of the workers who would be displaced, constitutes a synergistically more powerful political bloc than either alone.
For Taiwan Supply Chain
I believe the US is making significant strides in trying to create comparable chip fabs in the US, but this is going to take a few years to be on par even with continued government support. So this factor is more relevant in the next 5 years than it should be expected to be any time > 5 years from now.
For the R&D penalty
I’m not really convinced by this argument. I agree that the scaling factor for adding new researchers to a problem is quite difficult. But what if you just give each existing researcher an incredibly smart and powerful assistant? Seems to me that that situation is sufficiently different from naively adding more researchers as to perhaps not fit the same pattern.
Similarly, when the AI systems are researching on their own, I’m not convinced that it makes sense to model them as individual humans working separately. As Geoffrey Hinton has discussed in some recent interviews, AI has a big advantage here that humans don’t have. They can share weights, share knowledge nearly instantly with perfect fidelity. Thus, you can devise a system of AI workers who function as a sort of hivemind, constantly sharing ideas and insights with each other. This means they are closer to being ‘one giant researcher’ than being lots of individual humans. I don’t know how much this will make a difference, but I’m pretty sure the exact model from human researchers won’t fit very well.
On the topics of Data and Reasoning
I actually don’t think this does represent an upcoming roadblock. I actually think we are already in a data and compute overhang, and the thing holding us back is algorithmic development. I don’t think we are likely to get to AGI by scaling existing LLMs. I do think that existing LLMs will get far enough to be useful assistants to initially speed up ML R&D. But I think one of the effects of that speed up is going to be that researchers feel enabled, with the help of powerful assistants, to explore a broader range of hypotheses. Reading papers, extracting hypotheses, writing code to test these hypotheses, summarizing the results… these are all tasks which could be automated by sufficiently scaffolded LLMs not much better than today’s models.
I expect that the result of this will be discovering entirely new architectures which fundamentally have far more abstract reasoning ability, and don’t need nearly as much data or compute to train. If true, this will be a dangerous jump because it will actually be a discontinuous departure from the scaling pattern so far seen with transformer-based LLMs.
Hey Nathan, thanks for your comments. A few quick responses:
On Taiwan Supply Chain:
Agreed that US fabs don’t become a huge factor for a few years, even if everything “goes right” in their scale-up.
Important to note that even as the US fabs develop, other jurisdictions won’t pause their progress. Even with lots to be determined re: future innovation, lots has to “go right” to displace Taiwan from the pole position.
On R&D Penalty:
The “hive mind”/“One Giant Researcher” model might smooth out the inefficiency of communicating findings within research teams. However, this doesn’t solve the problem of different R&D teams working toward the same goals, thus “duplicating” their work. (Microsoft and Google won’t unite their “AI hive minds.” Nor will Apple and Huawei.)
Giving every researcher a super-smart assistant might help individual researcher productivity, but it doesn’t stop them from pursuing the same goals as their counterparts at other firms. It might accelerate progress without changing the parallelization penalty.
Concerns about private markets investment inefficiency still also contribute to a high parallelization penalty.
On Data and Reasoning:
“I actually think we are already in a data and compute overhang, and the thing holding us back is algorithmic development. I don’t think we are likely to get to AGI by scaling existing LLMs.”
If new breakthroughs in algorithm design solve the abstract reasoning challenge, then I agree! Models will need less data and compute to do more. I just think we’re major breakthrough or two away from that.
Davidson’s initial report builds off of a compute-centric model where “2020-era algorithms are powerful enough to reach AGI, if only provided enough compute.”
If you think we’re unlikely to get to AGI—or just solve the common sense problem—by scaling existing LLMs, then we will probably need more than just additional compute.
(I’d also push back on the idea that we’re already in a “data overhang” in many contexts. Both (1) robotics and (2) teaching specialized knowledge come to mind as domains where a shortage of quality data limits progress. But given our agreement above, that concern is downstream.)
So, I am absolutely on-board with the GDP Growth & Measurement critique, and the Taiwan Supply Chain problem.
For the GDP Growth & Measurement critique
I think you make good points here, but there are additional reason to expect even more slowdowns. For instance, I don’t think it’s quite as easy as the report makes it out to be to ramp up automation. I suspect that ramping up of novel technologies, like new sorts of robotics, will lead to novel bottlenecks in supply chains. Things which previously weren’t scarce will become (temporarily) scarce. There is a significant amount of inertia in ramping up novel verticals.
Another reason is that there’s going to be a lot of pushback from industry that will create delays. Investment into existing infrastructure creates strong incentives from wealthy powerful actors to lobby for not having their recently-built infrastructure obsoleted. This political force, plus the political force of the workers who would be displaced, constitutes a synergistically more powerful political bloc than either alone.
For Taiwan Supply Chain
I believe the US is making significant strides in trying to create comparable chip fabs in the US, but this is going to take a few years to be on par even with continued government support. So this factor is more relevant in the next 5 years than it should be expected to be any time > 5 years from now.
For the R&D penalty
I’m not really convinced by this argument. I agree that the scaling factor for adding new researchers to a problem is quite difficult. But what if you just give each existing researcher an incredibly smart and powerful assistant? Seems to me that that situation is sufficiently different from naively adding more researchers as to perhaps not fit the same pattern.
Similarly, when the AI systems are researching on their own, I’m not convinced that it makes sense to model them as individual humans working separately. As Geoffrey Hinton has discussed in some recent interviews, AI has a big advantage here that humans don’t have. They can share weights, share knowledge nearly instantly with perfect fidelity. Thus, you can devise a system of AI workers who function as a sort of hivemind, constantly sharing ideas and insights with each other. This means they are closer to being ‘one giant researcher’ than being lots of individual humans. I don’t know how much this will make a difference, but I’m pretty sure the exact model from human researchers won’t fit very well.
On the topics of Data and Reasoning
I actually don’t think this does represent an upcoming roadblock. I actually think we are already in a data and compute overhang, and the thing holding us back is algorithmic development. I don’t think we are likely to get to AGI by scaling existing LLMs. I do think that existing LLMs will get far enough to be useful assistants to initially speed up ML R&D. But I think one of the effects of that speed up is going to be that researchers feel enabled, with the help of powerful assistants, to explore a broader range of hypotheses. Reading papers, extracting hypotheses, writing code to test these hypotheses, summarizing the results… these are all tasks which could be automated by sufficiently scaffolded LLMs not much better than today’s models.
I expect that the result of this will be discovering entirely new architectures which fundamentally have far more abstract reasoning ability, and don’t need nearly as much data or compute to train. If true, this will be a dangerous jump because it will actually be a discontinuous departure from the scaling pattern so far seen with transformer-based LLMs.
For more details on the implications of this ‘new unspecified superior architecture, much better at reasoning and extrapolation from limited data’, I will direct you to this post by Thane: https://www.lesswrong.com/posts/HmQGHGCnvmpCNDBjc/current-ais-provide-nearly-no-data-relevant-to-agi-alignment
Hey Nathan, thanks for your comments. A few quick responses:
On Taiwan Supply Chain:
Agreed that US fabs don’t become a huge factor for a few years, even if everything “goes right” in their scale-up.
Important to note that even as the US fabs develop, other jurisdictions won’t pause their progress. Even with lots to be determined re: future innovation, lots has to “go right” to displace Taiwan from the pole position.
On R&D Penalty:
The “hive mind”/“One Giant Researcher” model might smooth out the inefficiency of communicating findings within research teams. However, this doesn’t solve the problem of different R&D teams working toward the same goals, thus “duplicating” their work. (Microsoft and Google won’t unite their “AI hive minds.” Nor will Apple and Huawei.)
Giving every researcher a super-smart assistant might help individual researcher productivity, but it doesn’t stop them from pursuing the same goals as their counterparts at other firms. It might accelerate progress without changing the parallelization penalty.
Concerns about private markets investment inefficiency still also contribute to a high parallelization penalty.
On Data and Reasoning:
If new breakthroughs in algorithm design solve the abstract reasoning challenge, then I agree! Models will need less data and compute to do more. I just think we’re major breakthrough or two away from that.
Davidson’s initial report builds off of a compute-centric model where “2020-era algorithms are powerful enough to reach AGI, if only provided enough compute.”
If you think we’re unlikely to get to AGI—or just solve the common sense problem—by scaling existing LLMs, then we will probably need more than just additional compute.
(I’d also push back on the idea that we’re already in a “data overhang” in many contexts. Both (1) robotics and (2) teaching specialized knowledge come to mind as domains where a shortage of quality data limits progress. But given our agreement above, that concern is downstream.)