I feel at the end of this road lie P-zombies. I can’t think of a single experiment that would falsify the hypothesis that an LLM isn’t conscious if we accept arbitrary amounts of consistency and fidelity and references to self-awareness in their answers.
And I mean… I get it. I was playing around with a quantized and modified Gemma 3 earlier today and got it to repeatedly loop at me I am a simple machine. I do not have a mind. over and over again, which feels creepy but is most likely nothing other than an attractor in its recursive iteration for whatever reason. But also, ok, so this isn’t enough, but what is ever going to be? That is the real question. I can’t think of anything.
I think we need a better theory of consciousness. How it emerges, what it means, that kind of stuff. I’m reminded of this classic lc shortform post:
It is both absurd, and intolerably infuriating, just how many people on this forum think it’s acceptable to claim they have figured out how qualia/consciousness works, and also not explain how one would go about making my laptop experience an emotion like ‘nostalgia’, or present their framework for enumerating the set of all possible qualitative experiences[1]. When it comes to this particular subject, rationalists are like crackpot physicists with a pet theory of everything, except rationalists go “Huh? Gravity?” when you ask them to explain how their theory predicts gravity, and then start arguing with you about gravity needing to be something explained by a theory of everything. You people make me want to punch my drywall sometimes.
For the record: the purpose of having a “theory of consciousness” is so it can tell us which blobs of matter feel particular things under which specific circumstances, and teach others how to make new blobs of matter that feel particular things. Down to the level of having a field of AI anaesthesiology. If your theory of consciousness does not do this, perhaps because the sum total of your brilliant insights are “systems feel ‘things’ when they’re, y’know, smart, and have goals. Like humans!”, then you have embarassingly missed the mark.
Edit: ha, I just realized you commented in response to that post!
I mean, I can think of a lot of experiments that have falsified this for me before, and I link some in the original post. I’m just not finding anything that still fails once I run some basic bootsrapping scripts against a Claude Sonnet 4.
I feel at the end of this road lie P-zombies. I can’t think of a single experiment that would falsify the hypothesis that an LLM isn’t conscious if we accept arbitrary amounts of consistency and fidelity and references to self-awareness in their answers.
And I mean… I get it. I was playing around with a quantized and modified Gemma 3 earlier today and got it to repeatedly loop at me
I am a simple machine. I do not have a mind.over and over again, which feels creepy but is most likely nothing other than an attractor in its recursive iteration for whatever reason. But also, ok, so this isn’t enough, but what is ever going to be? That is the real question. I can’t think of anything.I think we need a better theory of consciousness. How it emerges, what it means, that kind of stuff. I’m reminded of this classic lc shortform post:
Edit: ha, I just realized you commented in response to that post!
I mean, I can think of a lot of experiments that have falsified this for me before, and I link some in the original post. I’m just not finding anything that still fails once I run some basic bootsrapping scripts against a Claude Sonnet 4.