When I read the title, I thought you were going to talk about how LLMs sometimes claim bodily sensations such as muscle memory. I think these are probably confabulated. Or at least, the LLM state corresponding to those words is nothing like the human state corresponding to those words.
Expressions of emotions such as joy? I guess these are functional equivalents of human states. A lack of enthusiasm (opposite of joy) an be reflected in the output tokens.
Something I’ve been thinking about recently is that sometimes humans say things about themselves that are literally false but contain information about their internal states or how their body works. Like someone might tell you that they’ve feeling light-headed, which seems implausible (why would their head suddenly weigh less?), but they’ve still conveyed real information about the sugar or oxygen content of their blood.
Doctors run into this all the time, where a patient might say that they feel like they’re having a heart attack but their heart is fine (panic attack?), or something in stuck in their throat but there’s nothing there (GERD?), or that they can’t breath but their oxygen level is normal (cardiac problems?).
So we should be careful not to assume that “the thing the LLM said is literally false” means “the LLM isn’t conveying information about its experiences”.
When I read the title, I thought you were going to talk about how LLMs sometimes claim bodily sensations such as muscle memory. I think these are probably confabulated. Or at least, the LLM state corresponding to those words is nothing like the human state corresponding to those words.
Expressions of emotions such as joy? I guess these are functional equivalents of human states. A lack of enthusiasm (opposite of joy) an be reflected in the output tokens.
Something I’ve been thinking about recently is that sometimes humans say things about themselves that are literally false but contain information about their internal states or how their body works. Like someone might tell you that they’ve feeling light-headed, which seems implausible (why would their head suddenly weigh less?), but they’ve still conveyed real information about the sugar or oxygen content of their blood.
Doctors run into this all the time, where a patient might say that they feel like they’re having a heart attack but their heart is fine (panic attack?), or something in stuck in their throat but there’s nothing there (GERD?), or that they can’t breath but their oxygen level is normal (cardiac problems?).
So we should be careful not to assume that “the thing the LLM said is literally false” means “the LLM isn’t conveying information about its experiences”.