I think there’s a moderately likely limit to LLMs and other applications of the present machine-learning paradigm. Humans are powerful general intelligences because we can, individually and collectively, make use of different cognitive modules in a way that converges on coherence, rather than splitting off into different and conflicting subagents. Our brains seem to have stopped growing not when individuals hit diminishing intelligence returns, but when we got smart enough to network Dunbar-sized bands into low-latency collective intelligences, and then shrunk a bit when the Dunbar bands figured out how to network themselves—as The Flenser does in Vinge’s A Fire Upon the Deep—into larger, more differentiated, but higher-latency lower-bandwidth collective intelligences. While this obviously doesn’t guarantee that human+ level AGI will be nice to all other such GIs (that’s not true of humans either) it does suggest that if a superintelligence functions in the same modular-convergence ways humans do, it will tend to recognize similarly constituted coherent clusters that it can talk with as something analogous to near kin or other members (actual or potential) of its community, much like we do.
LLMs are a bit surprisingly useful, but they’re nowhere near being as inventive and enterprising as an Einstein or Feynman or Moses or a hunter-gatherer band (the ancestral ones who were investigating new tech and invented horticulture and animal domestication, not the contemporary atavists selected for civilizational refusenikhood), though maybe within a few decades of being able to do most of what a Von Neumann can do, if their development works out well enough; we’ve discovered that a lot of the “knowledge work” we pretended took real thought can be done by ghosts if we throw enough compute at them. That’s pretty cool, but it only looks “PhD level” because it turns out the marginal PhD doesn’t require anything a ghost can’t do.
By EoY 2030 I don’t expect LLMs to usually not mess up tasks like this one (scroll down a bit for the geometry fail), though any particular example that gets famous enough can get Goodharted even with minor perturbations via jerry-rigging enough non-LLM modules together. My subjective expectation is that they’ll still frequently fail the “strictly a word problem” version of such problems that require simple geometric reasoning about an object with multiple parts that isn’t a typical word-problem object.
I don’t expect them to be able to generate Dead Sea Scroll forgeries with predominantly novel content specified by the user, that hold up to good textual criticism, unless the good textual critics are all retired, dead, or marginalized. I don’t expect them to be able to write consistently in non-anachronistic idiomatic Elizabethan English, though possibly they’ll be able to write in Middle English.
Not sure these are strictly the “easiest” but they’re examples where I expect LLMs to underperform their vibe by a LOT, while still getting better at the things that they’re actually good at.
I think there’s a moderately likely limit to LLMs and other applications of the present machine-learning paradigm. Humans are powerful general intelligences because we can, individually and collectively, make use of different cognitive modules in a way that converges on coherence, rather than splitting off into different and conflicting subagents. Our brains seem to have stopped growing not when individuals hit diminishing intelligence returns, but when we got smart enough to network Dunbar-sized bands into low-latency collective intelligences, and then shrunk a bit when the Dunbar bands figured out how to network themselves—as The Flenser does in Vinge’s A Fire Upon the Deep—into larger, more differentiated, but higher-latency lower-bandwidth collective intelligences. While this obviously doesn’t guarantee that human+ level AGI will be nice to all other such GIs (that’s not true of humans either) it does suggest that if a superintelligence functions in the same modular-convergence ways humans do, it will tend to recognize similarly constituted coherent clusters that it can talk with as something analogous to near kin or other members (actual or potential) of its community, much like we do.
LLMs are a bit surprisingly useful, but they’re nowhere near being as inventive and enterprising as an Einstein or Feynman or Moses or a hunter-gatherer band (the ancestral ones who were investigating new tech and invented horticulture and animal domestication, not the contemporary atavists selected for civilizational refusenikhood), though maybe within a few decades of being able to do most of what a Von Neumann can do, if their development works out well enough; we’ve discovered that a lot of the “knowledge work” we pretended took real thought can be done by ghosts if we throw enough compute at them. That’s pretty cool, but it only looks “PhD level” because it turns out the marginal PhD doesn’t require anything a ghost can’t do.
what’s the easiest thing you think LLMs won’t be able to do in 5 years ie by EoY 2030? what about EoY 2026?
By EoY 2026 I don’t expect this to be a solved problem, though I expect people to find workarounds that involve lowered standards: https://benjaminrosshoffman.com/llms-for-language-learning/
By EoY 2030 I don’t expect LLMs to usually not mess up tasks like this one (scroll down a bit for the geometry fail), though any particular example that gets famous enough can get Goodharted even with minor perturbations via jerry-rigging enough non-LLM modules together. My subjective expectation is that they’ll still frequently fail the “strictly a word problem” version of such problems that require simple geometric reasoning about an object with multiple parts that isn’t a typical word-problem object.
I don’t expect them to be able to generate Dead Sea Scroll forgeries with predominantly novel content specified by the user, that hold up to good textual criticism, unless the good textual critics are all retired, dead, or marginalized. I don’t expect them to be able to write consistently in non-anachronistic idiomatic Elizabethan English, though possibly they’ll be able to write in Middle English.
Not sure these are strictly the “easiest” but they’re examples where I expect LLMs to underperform their vibe by a LOT, while still getting better at the things that they’re actually good at.