To clarify the “Bitter Lesson” example: the non-roundabout “direct” AI strategy is to use the “sweet shortcut” (h/t Gwern), by using existing human expert knowledge and trying to encode that into a computer. The roundabout strategy is to build a massive computing infrastructure first, which scaling requires. Even if no single group actually executed a strategy of “invent better computers and then do ML on them”, society as-a-whole did via the compute overhang.
To clarify the “Bitter Lesson” example: the non-roundabout “direct” AI strategy is to use the “sweet shortcut” (h/t Gwern), by using existing human expert knowledge and trying to encode that into a computer. The roundabout strategy is to build a massive computing infrastructure first, which scaling requires. Even if no single group actually executed a strategy of “invent better computers and then do ML on them”, society as-a-whole did via the compute overhang.