Fwiw I similarly still experience them to be bad at coming up with useful novel math research ideas, even as they’ve gotten much more competent at coding. Though they aren’t great at coding yet either.
However, I don’t think this ‘filling in the blanks’ is something fundamentally different in kind from ‘raw intelligence’. I don’t think there’s a hard boundary here. Anything that isn’t a literal lookup table is applying algorithms to extrapolate what it knows to new situations. Even something as minor as changing the tense of a memorised sentence is novel invention of a sort, just a tiny little bit. I think current llms can’t extrapolate as far as some humans yet, but the average distance they can extrapolate over seems to me to have increased over time. They’re still bad at coming up with novel math research ideas now, but three years ago they were much worse.
Separately from this, llms just know a lot of things most humans don’t, which can make them a value add to some intellectual tasks even if they can’t extrapolate the things they know very far.
Fwiw I similarly still experience them to be bad at coming up with useful novel math research ideas, even as they’ve gotten much more competent at coding. Though they aren’t great at coding yet either.
However, I don’t think this ‘filling in the blanks’ is something fundamentally different in kind from ‘raw intelligence’. I don’t think there’s a hard boundary here. Anything that isn’t a literal lookup table is applying algorithms to extrapolate what it knows to new situations. Even something as minor as changing the tense of a memorised sentence is novel invention of a sort, just a tiny little bit. I think current llms can’t extrapolate as far as some humans yet, but the average distance they can extrapolate over seems to me to have increased over time. They’re still bad at coming up with novel math research ideas now, but three years ago they were much worse.
Separately from this, llms just know a lot of things most humans don’t, which can make them a value add to some intellectual tasks even if they can’t extrapolate the things they know very far.