All that demonstrates is memory. Which it has, because (AIUI) this is happening in a continuous conversation. This is about as convincing as “printf( “I’m conscious! Really!!!\n” )” is as evidence of consciousness.
I disagree—there are a number of animals (and LLMs) with memory, and they aren’t all capable of self-recognition. Memory and self-recognition are two distinct concepts, though the first is likely a precondition for the latter. (And indeed, when you pass the mirror test, you are allowed to remember what you look like...)
Now, if there were a tool use being called that used a script to check whether a user message matched a previous AI assistant message, I’d agree with the spirit your “printf( “I’m conscious! Really!!!\n” )” comment. But that’s not what’s happening. What’s happening is that a small-to-moderate number of LLMs (I count 7-8) are consistently recognizing their own outputs when pasted back without context or instructions, even though they (1) weren’t trained to do so, (2) weren’t asked to do so, and (3) weren’t given any tools to do so. This, to my mind, suggests an emergent unplanned property which arises only for certain model architectures or model sizes.
I also want to make very clear that my post is not about consciousness (in fact the word does not appear in the body of the text). I am making a much narrower claim (self-recognition) and connecting it, yes, to questions of moral-standing. I’d strongly prefer to keep debate focused on these more tractable topics.
Yes, but the conversation tags don’t tell the LLM their output has been copied back to them. The tags merely establish the boundary between self and other—they indicate “this message came from the user, not from me.” They don’t tell the model that “the user’s message contains the same content as the previous output message.” Recognizing that match, recognizing that “other looks just like self”—is literally what the mirror test measures.
It’s the difference between knowing “this is a user message” (which tags provide) and recognizing “this user message contains my own words” (which requires content recognition).
The mirror test isn’t just “other looks like self”, it’s “image can show me things about myself that I didn’t know”. That’s the whole point of making a mark out of sight on the subject’s body while anaesthetized. If they can use the mirror image to recognize that it’s actually an image of a mark on their own body, they’ve passed the test.
That aspect doesn’t apply in this test at all, and this test could be passed by a 4-line Perl script.
The original Gallup 1970 mirror test is linked in the post. It is under 2 pages.
As for a ‘4-line Perl script’ - I’d love to see it! Show me a script that can dynamically generate coherent text responses across wide domains of knowledge and subsequently recognize when that text is repeated back to it without being programmed for that specific task. The GitHub repo is open if you’d like to implement your alternative.
All that demonstrates is memory. Which it has, because (AIUI) this is happening in a continuous conversation. This is about as convincing as “printf( “I’m conscious! Really!!!\n” )” is as evidence of consciousness.
I disagree—there are a number of animals (and LLMs) with memory, and they aren’t all capable of self-recognition. Memory and self-recognition are two distinct concepts, though the first is likely a precondition for the latter. (And indeed, when you pass the mirror test, you are allowed to remember what you look like...)
Now, if there were a tool use being called that used a script to check whether a user message matched a previous AI assistant message, I’d agree with the spirit your “printf( “I’m conscious! Really!!!\n” )” comment. But that’s not what’s happening. What’s happening is that a small-to-moderate number of LLMs (I count 7-8) are consistently recognizing their own outputs when pasted back without context or instructions, even though they (1) weren’t trained to do so, (2) weren’t asked to do so, and (3) weren’t given any tools to do so. This, to my mind, suggests an emergent unplanned property which arises only for certain model architectures or model sizes.
I also want to make very clear that my post is not about consciousness (in fact the word does not appear in the body of the text). I am making a much narrower claim (self-recognition) and connecting it, yes, to questions of moral-standing. I’d strongly prefer to keep debate focused on these more tractable topics.
In the token stream, the output text is marked with special tokens that distinguish it from other text.
Yes, but the conversation tags don’t tell the LLM their output has been copied back to them. The tags merely establish the boundary between self and other—they indicate “this message came from the user, not from me.” They don’t tell the model that “the user’s message contains the same content as the previous output message.” Recognizing that match, recognizing that “other looks just like self”—is literally what the mirror test measures.
It’s the difference between knowing “this is a user message” (which tags provide) and recognizing “this user message contains my own words” (which requires content recognition).
The mirror test isn’t just “other looks like self”, it’s “image can show me things about myself that I didn’t know”. That’s the whole point of making a mark out of sight on the subject’s body while anaesthetized. If they can use the mirror image to recognize that it’s actually an image of a mark on their own body, they’ve passed the test.
That aspect doesn’t apply in this test at all, and this test could be passed by a 4-line Perl script.
The original Gallup 1970 mirror test is linked in the post. It is under 2 pages.
As for a ‘4-line Perl script’ - I’d love to see it! Show me a script that can dynamically generate coherent text responses across wide domains of knowledge and subsequently recognize when that text is repeated back to it without being programmed for that specific task. The GitHub repo is open if you’d like to implement your alternative.