Disagree with these. Humans don’t automatically make all the facts in their head cohere. I think its plausible that they’re worse at humans at doing this. But that seems insufficient for making a discrete demarcation. For example:
and then make some totally different and incompatible assumption about the symbol later on in the proof, as though it means something totally different.
This happens pretty often with humans actually? Like one of the most common ways people (compsci undergrads and professional mathematicians alike) make errors in proofs is like
Now we’ve proven that if object G has properties A,B,C,D, G also has property P … [steps] … And as we know, object G’ has properties A,B,C therefore has property P...
I agree that there are ways LLMs understanding is shallower than humans, but from my PoV, a lot of that impression comes from.
When you use the models you consistently restart and rewind them. This means their flaws are laid much more bare than when talking to humans. Like, if we were having a debate about, for example, the degree to which LLMs can be said to understand things, and we had that debate 100 times, but my memory was wiped between each debate, and yours kept, I’m pretty sure I’d look very stupid and shallow. Not that different from a chatbot.
Lack of continual learning type things? Like if in a conversation, you say something that makes me realize my understanding of a concept is incomplete or subtly inconsistent, I might in the background update my understanding and reevaluate whether my claims still make sense in lieu of that revised understanding. LLMs seem to have a hard time doing this. But that seems like a meaningfully different problem from “not understanding” stuff
Having a lot of knowledge. Like if you met a human who could solve algebraic geometry problems, you’d assume they were pretty smart. And if they were unable to like, connect their headset to their phone via bluetooth despite trying for a week, and nothing being wrong with their phone/headset, you’d be seriously confused/surprised, and think they probably had some mental disorder, or were just messing with you. But like, LLMs, are more like a 60IQ human who knows a huge amount of things, and is unusually good at manipulating symbols and dealing with (from a human perspective) abstraction. And like, that a 60IQ human will forget what they meant with a term, or will not have totally clear mental images of the domain they’re dealing with, would maybe not be that surprising.
Disagree with these. Humans don’t automatically make all the facts in their head cohere.
Hm, do you see the OP as arguing that it happens “automatically”? My reading was more like that it happens “eventually, if motivated to figure it out” and that we don’t know how to “motivate” LLMs to be good at this in an efficient way (yet).
people (compsci undergrads and professional mathematicians alike) make errors in proofs
Sure, and would you hire those people and rely on them to do a good job BEFORE they learn better?
Disagree with these. Humans don’t automatically make all the facts in their head cohere. I think its plausible that they’re worse at humans at doing this. But that seems insufficient for making a discrete demarcation. For example:
This happens pretty often with humans actually? Like one of the most common ways people (compsci undergrads and professional mathematicians alike) make errors in proofs is like
I agree that there are ways LLMs understanding is shallower than humans, but from my PoV, a lot of that impression comes from.
When you use the models you consistently restart and rewind them. This means their flaws are laid much more bare than when talking to humans. Like, if we were having a debate about, for example, the degree to which LLMs can be said to understand things, and we had that debate 100 times, but my memory was wiped between each debate, and yours kept, I’m pretty sure I’d look very stupid and shallow. Not that different from a chatbot.
Lack of continual learning type things? Like if in a conversation, you say something that makes me realize my understanding of a concept is incomplete or subtly inconsistent, I might in the background update my understanding and reevaluate whether my claims still make sense in lieu of that revised understanding. LLMs seem to have a hard time doing this. But that seems like a meaningfully different problem from “not understanding” stuff
Having a lot of knowledge. Like if you met a human who could solve algebraic geometry problems, you’d assume they were pretty smart. And if they were unable to like, connect their headset to their phone via bluetooth despite trying for a week, and nothing being wrong with their phone/headset, you’d be seriously confused/surprised, and think they probably had some mental disorder, or were just messing with you. But like, LLMs, are more like a 60IQ human who knows a huge amount of things, and is unusually good at manipulating symbols and dealing with (from a human perspective) abstraction. And like, that a 60IQ human will forget what they meant with a term, or will not have totally clear mental images of the domain they’re dealing with, would maybe not be that surprising.
Hm, do you see the OP as arguing that it happens “automatically”? My reading was more like that it happens “eventually, if motivated to figure it out” and that we don’t know how to “motivate” LLMs to be good at this in an efficient way (yet).
Sure, and would you hire those people and rely on them to do a good job BEFORE they learn better?