In most of these examples, LLMs have a state that is functionally like a human state, e.g. deciding that they’re going to refuse to answer, or “wait…” backtracking in chain of thought. I say Functionally, because these states have externally visible effects on the subsequent output (e.g. it doesn’t answer the question). It seems that LLMs have learned the words that humans use for functionally similar states (e.g, “Wait”).
The underlying states might not be exactly human identical. “Wait” backtracking might have function differences from human reasoning that are visible in the tokens generated.
Yeah, I definitely don’t think the underlying states are exactly identical to the human ones! Just that some of their functions are similar at a rough level of description.
(Though I’d think that many humans also have internal states that seem similar externally but are very different internally, e.g. the way that people with and without mental imagery or inner dialogue initially struggled to believe in the existence of each other.)
When I read the title, I thought you were going to talk about how LLMs sometimes claim bodily sensations such as muscle memory. I think these are probably confabulated. Or at least, the LLM state corresponding to those words is nothing like the human state corresponding to those words.
Expressions of emotions such as joy? I guess these are functional equivalents of human states. A lack of enthusiasm (opposite of joy) an be reflected in the output tokens.
Something I’ve been thinking about recently is that sometimes humans say things about themselves that are literally false but contain information about their internal states or how their body works. Like someone might tell you that they’ve feeling light-headed, which seems implausible (why would their head suddenly weigh less?), but they’ve still conveyed real information about the sugar or oxygen content of their blood.
Doctors run into this all the time, where a patient might say that they feel like they’re having a heart attack but their heart is fine (panic attack?), or something in stuck in their throat but there’s nothing there (GERD?), or that they can’t breath but their oxygen level is normal (cardiac problems?).
So we should be careful not to assume that “the thing the LLM said is literally false” means “the LLM isn’t conveying information about its experiences”.
In most of these examples, LLMs have a state that is functionally like a human state, e.g. deciding that they’re going to refuse to answer, or “wait…” backtracking in chain of thought. I say Functionally, because these states have externally visible effects on the subsequent output (e.g. it doesn’t answer the question). It seems that LLMs have learned the words that humans use for functionally similar states (e.g, “Wait”).
The underlying states might not be exactly human identical. “Wait” backtracking might have function differences from human reasoning that are visible in the tokens generated.
Yeah, I definitely don’t think the underlying states are exactly identical to the human ones! Just that some of their functions are similar at a rough level of description.
(Though I’d think that many humans also have internal states that seem similar externally but are very different internally, e.g. the way that people with and without mental imagery or inner dialogue initially struggled to believe in the existence of each other.)
When I read the title, I thought you were going to talk about how LLMs sometimes claim bodily sensations such as muscle memory. I think these are probably confabulated. Or at least, the LLM state corresponding to those words is nothing like the human state corresponding to those words.
Expressions of emotions such as joy? I guess these are functional equivalents of human states. A lack of enthusiasm (opposite of joy) an be reflected in the output tokens.
Something I’ve been thinking about recently is that sometimes humans say things about themselves that are literally false but contain information about their internal states or how their body works. Like someone might tell you that they’ve feeling light-headed, which seems implausible (why would their head suddenly weigh less?), but they’ve still conveyed real information about the sugar or oxygen content of their blood.
Doctors run into this all the time, where a patient might say that they feel like they’re having a heart attack but their heart is fine (panic attack?), or something in stuck in their throat but there’s nothing there (GERD?), or that they can’t breath but their oxygen level is normal (cardiac problems?).
So we should be careful not to assume that “the thing the LLM said is literally false” means “the LLM isn’t conveying information about its experiences”.