Because it’s been experimentally verified that what they’re internally doing doesn’t match their verbal descriptions (not that there was really any reason to believe it would); see the section in this post (or in the associated paper for slightly more detail) about mental math, where Claude claims to perform addition in the same fashion humans do despite interpretability revealing otherwise.
Funny, I could have told you most of that just from talking to Claude. Having an official paper confirm that understanding really just strengths my understanding that you CAN get useful answers out of Claude, you just need to ask intelligent questions and not believe the first answer you get.
It’s the same error-correction process I’d use on a human: six year olds cannot reliably produce accurate answers on how they do arithmetic, but you can still figure it out pretty easily by just talking to them. I don’t think neurology has added much to our educational pedagogy (although do correct me if I’m missing something big there)
Because it’s been experimentally verified that what they’re internally doing doesn’t match their verbal descriptions (not that there was really any reason to believe it would); see the section in this post (or in the associated paper for slightly more detail) about mental math, where Claude claims to perform addition in the same fashion humans do despite interpretability revealing otherwise.
Funny, I could have told you most of that just from talking to Claude. Having an official paper confirm that understanding really just strengths my understanding that you CAN get useful answers out of Claude, you just need to ask intelligent questions and not believe the first answer you get.
It’s the same error-correction process I’d use on a human: six year olds cannot reliably produce accurate answers on how they do arithmetic, but you can still figure it out pretty easily by just talking to them. I don’t think neurology has added much to our educational pedagogy (although do correct me if I’m missing something big there)