I don’t think Paul is saying the same thing as me. My wording was bad, sorry.
When I said “the AIs that everyone is using and everyone is talking about”, I should have said “the AIs that are receiving a very large share of overall attention and investment by the ML research community”. (I just went back and edited the original.)
As of today (2022), large language models are “the AIs that are receiving a very large share of overall attention and investment by the ML research community”. But they are not having a “large impact on the world” by Paul’s definition. For example, the current contribution of large language models to global GDP is ≈0%.
The question of whether an AI approach is “receiving a very large share of overall attention and investment by the ML research community” is very important because:
if yes, we expect low-hanging fruit to be rapidly picked, after which we expect incremental smaller advances perpetually, and we expect state-of-the-art models to be using roughly the maximum amount of compute that is at all possible to use.
if no (i.e. if an AI approach is comparatively a bit of a backwater, like say model-based RL or probabilistic programming today), we should be less surprised by (for example) a flurry of very impactful advances within a short period of time, while most people aren’t paying attention, and then bam, we have a recipe for a superhuman AGI that can be trained on a university GPU cluster.
I suspect that LLMs are going to be put to more and more practical use in the near future. I just did a search on “AI and legal briefs” and came up with ads and articles about “prediction based” systems to help lawyers prepare legal briefs. I assume “prediction based” means LLM.
I don’t think Paul is saying the same thing as me. My wording was bad, sorry.
When I said “the AIs that everyone is using and everyone is talking about”, I should have said “the AIs that are receiving a very large share of overall attention and investment by the ML research community”. (I just went back and edited the original.)
As of today (2022), large language models are “the AIs that are receiving a very large share of overall attention and investment by the ML research community”. But they are not having a “large impact on the world” by Paul’s definition. For example, the current contribution of large language models to global GDP is ≈0%.
The question of whether an AI approach is “receiving a very large share of overall attention and investment by the ML research community” is very important because:
if yes, we expect low-hanging fruit to be rapidly picked, after which we expect incremental smaller advances perpetually, and we expect state-of-the-art models to be using roughly the maximum amount of compute that is at all possible to use.
if no (i.e. if an AI approach is comparatively a bit of a backwater, like say model-based RL or probabilistic programming today), we should be less surprised by (for example) a flurry of very impactful advances within a short period of time, while most people aren’t paying attention, and then bam, we have a recipe for a superhuman AGI that can be trained on a university GPU cluster.
Ok I see what you mean, thanks for clarifying.
I suspect that LLMs are going to be put to more and more practical use in the near future. I just did a search on “AI and legal briefs” and came up with ads and articles about “prediction based” systems to help lawyers prepare legal briefs. I assume “prediction based” means LLM.