Sure, I think that’s an important consideration and I pin some hopes of slowdown on it.
I’ll just make my standard caveat that there are plenty of routes to improvement that don’t depend strictly on scaling. Particularly salient is improved memory and metacognition/executive function. Both areas of active research, documented in those posts, so both are being improved independently of gains from scaling.
This isn’t likely to produce a step change, since progress on both is in amount not kind, but it’s another factor that will drive improvements in competence and capabilities, that’s not bottlenecked by compute.
Sure, I think that’s an important consideration and I pin some hopes of slowdown on it.
I’ll just make my standard caveat that there are plenty of routes to improvement that don’t depend strictly on scaling. Particularly salient is improved memory and metacognition/executive function. Both areas of active research, documented in those posts, so both are being improved independently of gains from scaling.
This isn’t likely to produce a step change, since progress on both is in amount not kind, but it’s another factor that will drive improvements in competence and capabilities, that’s not bottlenecked by compute.