So my general take is that the thing worth labeling “consciousness” is closed-loop feedback systems. Not everyone agrees, but since I need some definition of consciousness to reply, I’ll go forward with that.
Under this definition, LLMs are not strictly conscious because they don’t form a closed loop. However, the LLM-human system might be said to be conscious because it does.
This is pretty unintuitive because sometimes what people mean by “conscious” is “directed awareness”, in which case an LLM is definitely not conscious, and to any extent the LLM-human system is conscious, it seems to be conscious because the human is conscious, so we can’t really say the LLM part is conscious.
Non-LLM AI do sometimes meet my definition for consciousness.
(I think there’s also something to say here about how LLMs work and how it making claims about being conscious is small evidence of actual consciousness.)
Then I think you are only redefining the word, not asserting something about the thing it previously meant.
This looks like an example of an argument that goes, consciousness has a property P, therefore everything with property P is conscious. The P here is “closed-loop feedback”. Panpsychists use P = existence. Others might use P = making models of the world, P = making models of themselves, P = capable of suffering, or P = communicating with others. Often this is accompanied by phrases like “in a sense” or “to some degree”.
Why do you choose the particular P that you do? What thing is being claimed of all P objects, that goes beyond merely being P, when you say, these too are conscious?
I wouldn’t say I’m redefining “consciousness” exactly. Rather, “consciousness” has a vague definition, and I’m offering a more precise definition in terms of what I think reasonably captures the core of what we intend to mean by “consciousness” and is consistent with the world as we find it. Unfortunately, since “consciousness” is vaguely defined, people disagree on intuitions about what the core of it is.
Personally I try to just avoid using the term “consciousness” these days because it’s so confusing, but other people like to say it, and closed loop feedback is how I make sense of it.
As to why I think closed-loop feedback is the right way to think about “consciousness”, I included some links to things I wrote a while ago in a sibling comment reply.
Define a closed-loop feedback system? Human six year olds get inputs from both other humans, and the external world—they don’t exist in a platonic void.
my general take is that the thing worth labeling “consciousness” is closed-loop feedback systems
Have you written more about this anywhere? As written, this seems way too broad (depending on what you mean by this, I suspect I can concoct some automated thermostat+environment system which satisfies the definition but would strike people as a ridiculous example of consciousness).
Yes-ish, but it’s all kind of old and I don’t like how much metaphysics I mixed into those posts. So you can read them, but you need to read them in the sense of me working out ideas more than me getting all the details right (though I endorse the key idea about what I believe is the thing worth calling “consciousness”).
So my general take is that the thing worth labeling “consciousness” is closed-loop feedback systems. Not everyone agrees, but since I need some definition of consciousness to reply, I’ll go forward with that.
Under this definition, LLMs are not strictly conscious because they don’t form a closed loop. However, the LLM-human system might be said to be conscious because it does.
This is pretty unintuitive because sometimes what people mean by “conscious” is “directed awareness”, in which case an LLM is definitely not conscious, and to any extent the LLM-human system is conscious, it seems to be conscious because the human is conscious, so we can’t really say the LLM part is conscious.
Non-LLM AI do sometimes meet my definition for consciousness.
(I think there’s also something to say here about how LLMs work and how it making claims about being conscious is small evidence of actual consciousness.)
Even thermostats? Or is that a necessary but not sufficient condition?
Yes, even thermostats.
Then I think you are only redefining the word, not asserting something about the thing it previously meant.
This looks like an example of an argument that goes, consciousness has a property P, therefore everything with property P is conscious. The P here is “closed-loop feedback”. Panpsychists use P = existence. Others might use P = making models of the world, P = making models of themselves, P = capable of suffering, or P = communicating with others. Often this is accompanied by phrases like “in a sense” or “to some degree”.
Why do you choose the particular P that you do? What thing is being claimed of all P objects, that goes beyond merely being P, when you say, these too are conscious?
I wouldn’t say I’m redefining “consciousness” exactly. Rather, “consciousness” has a vague definition, and I’m offering a more precise definition in terms of what I think reasonably captures the core of what we intend to mean by “consciousness” and is consistent with the world as we find it. Unfortunately, since “consciousness” is vaguely defined, people disagree on intuitions about what the core of it is.
Personally I try to just avoid using the term “consciousness” these days because it’s so confusing, but other people like to say it, and closed loop feedback is how I make sense of it.
As to why I think closed-loop feedback is the right way to think about “consciousness”, I included some links to things I wrote a while ago in a sibling comment reply.
Define a closed-loop feedback system? Human six year olds get inputs from both other humans, and the external world—they don’t exist in a platonic void.
https://en.wikipedia.org/wiki/Closed-loop_controller
Have you written more about this anywhere? As written, this seems way too broad (depending on what you mean by this, I suspect I can concoct some automated thermostat+environment system which satisfies the definition but would strike people as a ridiculous example of consciousness).
Yes-ish, but it’s all kind of old and I don’t like how much metaphysics I mixed into those posts. So you can read them, but you need to read them in the sense of me working out ideas more than me getting all the details right (though I endorse the key idea about what I believe is the thing worth calling “consciousness”).
https://www.lesswrong.com/posts/ERp5ERYAaXFz4c8uF/ai-alignment-and-phenomenal-consciousness
and
https://www.lesswrong.com/posts/M7Z5sm6KoukNpF3SD/form-and-feedback-in-phenomenology
Curious to hear a few examples for this? Would something like AlphaGo meet the definition?