The question this addresses is whether LLMs can create new knowledge. The answer is “that’s irrelevant”.
Your framing seems to equivocate over current LLMs, future LLMs, and future AI of all types. That’s exactly what the public debate does, and it creates a flaming mess.
I’m becoming concerned that too many in the safety community are making this same mistake, and thereby misunderstanding and underestimating the near-term danger.
I think there’s a good point to be made about the cognitive limitations of LLMs. I doubt they can achieve AGI on their own.
But they don’t have to, so whether they can is irrelevant.
If you look at how humans create knowledge, we are using a combination of techniques and brain systems that LLMs cannot employ. Those include continuous, self-directed learning; episodic memory as one aid to that learning; cognitive control, to organize and direct that learning; and sensory and motor systems to carry out experiments to direct that learning.
All of those are conceptually straightforward to add to LLMs (good executive function/cognitive control for planning is less obviously straightforward, but I think it may be surprsingly easy to leverage LLMs “intelligence” to do it well).
See my Capabilities and alignment of LLM cognitive architectures for expansions on those arguments. I’ve been reluctant to publish more, but I think these ideas are fairly obvious once someone actually sits down to create agents that expand on LLM capabilities, so I think getting the alignment community thinking about this correctly is more important than a tiny slowdown in reaching x-risk capable AGI through this route.
(BTW human artistic creativity uses that same set of cognitive capabilities in different ways, so same answer to “can LLMs be true artists”).
I don’t think I disagree with you on anything; my point is more “what does creating new knowledge mean?”. For example, the difference between interpolation and extrapolation might be a rigorous way of framing it. Someone else posted a LeCun paper on that here; he found that extrapolation is the regime in which most ML systems work and assumes that the same must be true of deep learning ones. But for example if there was a phase transition of some kind in the learning process that makes some systems move to an interpolation regime, that could explain things. Overall I agree that none of this should be a fundamental difference with human cognition. It could be a current one, but it would at least be possible to overcome in principle. Or LLMs could already be in this new regime, since after all, not like anyone checked yet (honestly though, it might not be too hard to do so, and we should probably try).
The question this addresses is whether LLMs can create new knowledge. The answer is “that’s irrelevant”.
Your framing seems to equivocate over current LLMs, future LLMs, and future AI of all types. That’s exactly what the public debate does, and it creates a flaming mess.
I’m becoming concerned that too many in the safety community are making this same mistake, and thereby misunderstanding and underestimating the near-term danger.
I think there’s a good point to be made about the cognitive limitations of LLMs. I doubt they can achieve AGI on their own.
But they don’t have to, so whether they can is irrelevant.
If you look at how humans create knowledge, we are using a combination of techniques and brain systems that LLMs cannot employ. Those include continuous, self-directed learning; episodic memory as one aid to that learning; cognitive control, to organize and direct that learning; and sensory and motor systems to carry out experiments to direct that learning.
All of those are conceptually straightforward to add to LLMs (good executive function/cognitive control for planning is less obviously straightforward, but I think it may be surprsingly easy to leverage LLMs “intelligence” to do it well).
See my Capabilities and alignment of LLM cognitive architectures for expansions on those arguments. I’ve been reluctant to publish more, but I think these ideas are fairly obvious once someone actually sits down to create agents that expand on LLM capabilities, so I think getting the alignment community thinking about this correctly is more important than a tiny slowdown in reaching x-risk capable AGI through this route.
(BTW human artistic creativity uses that same set of cognitive capabilities in different ways, so same answer to “can LLMs be true artists”).
I don’t think I disagree with you on anything; my point is more “what does creating new knowledge mean?”. For example, the difference between interpolation and extrapolation might be a rigorous way of framing it. Someone else posted a LeCun paper on that here; he found that extrapolation is the regime in which most ML systems work and assumes that the same must be true of deep learning ones. But for example if there was a phase transition of some kind in the learning process that makes some systems move to an interpolation regime, that could explain things. Overall I agree that none of this should be a fundamental difference with human cognition. It could be a current one, but it would at least be possible to overcome in principle. Or LLMs could already be in this new regime, since after all, not like anyone checked yet (honestly though, it might not be too hard to do so, and we should probably try).