It sounds to me like you are describing a world where AIs don’t fully automate AI R&D but can maybe 98% automate AI R&D. As in, these AIs aren’t able to automate the task of generally making AIs smarter and finding new paradigms/methods, but can automate 99%+ of things within the current paradigm / approach. I tend to think the true extremes/limits of the current paradigm would be very extreme (even if weak in some ways) such that you’d be able to bootstrap if you actually reached this extreme. I also think that getting to full automation of the current paradigm would effectively require AIs to at least be OK at figuring out significant advances in general. (In the same way as I think that for AIs to be able to automate the job of research engineer at AI companies, they’d probably need to be at least OK at automating most SWE jobs in general.)
No, the AIs do fully automate R&D, AI and otherwise. But the speed with which they do R&D depends not just on the speed of token generation, but also on the speed at which they learn deep skills, and the latter is much lower for LLMs built with the current methods (they only learn deep skills in new model releases).
Token generation speed gives an anchor of maybe 200x serial speedup compared to humans, plus very scalable parallel labor, minus real world constraints from needing experimental feedback (which don’t even apply to some forms of theory). But the need to learn deep skills makes this anchor misleading for the forms of labor that require many serial steps of learning new things that couldn’t be all learned in advance (in parallel). This is mostly R&D, and there the anchor of massive speedup just doesn’t apply, even though the work does get fully automated. Thus this is AGI, but it’s not a very fast AGI, because the AGI is the automated model-building process (that creates successive versions of LLMs), rather than the token-generating LLMs themselves.
So for some R&D things, these self-building AIs might be somewhat faster than humans, and for others slower. They are AGI in the same sense as humanity, capable of eventually causing a takeoff, but not very fast at getting there.
It sounds to me like you are describing a world where AIs don’t fully automate AI R&D but can maybe 98% automate AI R&D. As in, these AIs aren’t able to automate the task of generally making AIs smarter and finding new paradigms/methods, but can automate 99%+ of things within the current paradigm / approach. I tend to think the true extremes/limits of the current paradigm would be very extreme (even if weak in some ways) such that you’d be able to bootstrap if you actually reached this extreme. I also think that getting to full automation of the current paradigm would effectively require AIs to at least be OK at figuring out significant advances in general. (In the same way as I think that for AIs to be able to automate the job of research engineer at AI companies, they’d probably need to be at least OK at automating most SWE jobs in general.)
No, the AIs do fully automate R&D, AI and otherwise. But the speed with which they do R&D depends not just on the speed of token generation, but also on the speed at which they learn deep skills, and the latter is much lower for LLMs built with the current methods (they only learn deep skills in new model releases).
Token generation speed gives an anchor of maybe 200x serial speedup compared to humans, plus very scalable parallel labor, minus real world constraints from needing experimental feedback (which don’t even apply to some forms of theory). But the need to learn deep skills makes this anchor misleading for the forms of labor that require many serial steps of learning new things that couldn’t be all learned in advance (in parallel). This is mostly R&D, and there the anchor of massive speedup just doesn’t apply, even though the work does get fully automated. Thus this is AGI, but it’s not a very fast AGI, because the AGI is the automated model-building process (that creates successive versions of LLMs), rather than the token-generating LLMs themselves.
So for some R&D things, these self-building AIs might be somewhat faster than humans, and for others slower. They are AGI in the same sense as humanity, capable of eventually causing a takeoff, but not very fast at getting there.