If automated learning of deep skills is a crux for AGI, then automating AI R&D becomes more relevant in the sense of routine application of the current methods. Routine AI R&D is then a milestone on the way to AGI, while frontier AI R&D is merely an indicator that AGI has been achieved, not relevant to AGI timelines. This milestone plausibly doesn’t need any new breakthroughs or even significant scaling.
The distinction between the capability for carrying out tasks and the depth of skills that can be learned thus becomes the distinction between manual training of AIs by humans with routine application of current methods (the current situation) and its automation (the milestone where AIs apply those same methods on their own to train new skills into the next version of the AIs). Routine AI R&D is itself merely a collection of mostly fixed skills, doesn’t involve acquisition of new deep skills, and so probably can be trained into AIs by humans using routine AI R&D. In contrast, frontier AI R&D can require developing and mastering novel deep skills or theories, and so can’t be done automatically by an AI that is incapable of automatically learning deep skills.
It’s plausible this on its own is sufficient for AGI in the sense of the full automation of civilization. This is something that can be vaguely predicted with capability benchmarks (unlike the more direct attacks on remaining hobblings), and improves with inputs to capabilities such as the amount of compute.
Thus I think when people talk about imminent automated AI R&D or the achievement of AGI (in 2026-2027), it could be in the sense of routine AI R&D, which then starts the flywheel of accumulating novel deep skills in consecutive versions of AIs, a process that builds towards AGI-level capabilities, including frontier AI R&D, without requiring any new unhobblings or breakthroughs invented by humans as prerequisites. This milestone of automated routine AI R&D seems disturbingly closer than an AI that starts doing frontier AI R&D directly, but it plausibly quickly leads to the same place.
If automated learning of deep skills is a crux for AGI, then automating AI R&D becomes more relevant in the sense of routine application of the current methods. Routine AI R&D is then a milestone on the way to AGI, while frontier AI R&D is merely an indicator that AGI has been achieved, not relevant to AGI timelines. This milestone plausibly doesn’t need any new breakthroughs or even significant scaling.
The distinction between the capability for carrying out tasks and the depth of skills that can be learned thus becomes the distinction between manual training of AIs by humans with routine application of current methods (the current situation) and its automation (the milestone where AIs apply those same methods on their own to train new skills into the next version of the AIs). Routine AI R&D is itself merely a collection of mostly fixed skills, doesn’t involve acquisition of new deep skills, and so probably can be trained into AIs by humans using routine AI R&D. In contrast, frontier AI R&D can require developing and mastering novel deep skills or theories, and so can’t be done automatically by an AI that is incapable of automatically learning deep skills.
It’s plausible this on its own is sufficient for AGI in the sense of the full automation of civilization. This is something that can be vaguely predicted with capability benchmarks (unlike the more direct attacks on remaining hobblings), and improves with inputs to capabilities such as the amount of compute.
Thus I think when people talk about imminent automated AI R&D or the achievement of AGI (in 2026-2027), it could be in the sense of routine AI R&D, which then starts the flywheel of accumulating novel deep skills in consecutive versions of AIs, a process that builds towards AGI-level capabilities, including frontier AI R&D, without requiring any new unhobblings or breakthroughs invented by humans as prerequisites. This milestone of automated routine AI R&D seems disturbingly closer than an AI that starts doing frontier AI R&D directly, but it plausibly quickly leads to the same place.