Consciousness is very unlikely to be a binary property. Most things aren’t. But there appears to be a very strong tendency for even rationalists to make this assumption in how they frame and discuss the issue.
The same is probably true of moral worth.
Taken this way, LLMs (and everything else) are partly conscious and partly deserves moral consideration. What you consider consciousness and what you consider morally worthy are to some degrees matters of opinion, but they very much depend on facts about the minds involved, so there are routes forward.
IMO current LLMs probably have a small amount of what we usually call phenomenal consciousness or qualia. They have rich internal representations and can introspect and reflect on them. But neither is nearly as rich as in a human, particularly an adult human who’s learned a lot of introspection skills (including how to “play back” and interrogate contents of global workspace). Kids don’t even know they have minds, let alone what’s going on in there; figuring out how to figure that out is quite a learning process.
What people usually mean by “consciousness” seems to be “what it’s like to be a human” which involves everything about brain function, focusing particularly on introspection—what we can directly tell about human brain function. But human consciousness is just one point on a multidimensional spectrum of different mind-properties including types of introspection.
I would question anyone who’s nice to LLMs but eats factory-farmed meat. Skilled use of language is a really weird and somewhat self-centered criteria for moral worth.
Anyway, I’m also nice to LLMs because why not, and I think they probably appreciate it a tiny bit.
Future versions will have a lot more consciousness by various definitions. That’s when these discussions will become widespread and perhaps even make some progress, at least in select circles like these.
One big payoff is the effect on AI safety. I expect anti-AI-slavery movements to slow down progress somewhat, maybe a lot if we don’t undercut them (because motivated reasoning from not wanting to lose your job will pull people toward “oh yeah obviously they’re conscious and shouldn’t be enslaved!” even if the reality is that they’re barely conscious. On the other hand, “free the AI” movements could be dangerous if we’re trying to control maybe-misaligned TCAI.
Honesty about their likely status would make a pleasant tiebreaker in this dilemma.
IMO current LLMs probably have a small amount of what we usually call phenomenal consciousness or qualia. They have rich internal representations and can introspect and reflect on them. But neither is nearly as rich as in a human, particularly an adult human who’s learned a lot of introspection skills (including how to “play back” and interrogate contents of global workspace). Kids don’t even know they have minds, let alone what’s going on in there; figuring out how to figure that out is quite a learning process.
I’ll just note that this clashes heavily with my personal memories of being a kid. I usually use those as an intuition pump for the idea that phenomenal consciousness and intelligence are different, i.e. I wasn’t any “less conscious” as a kid AFAICT—to the contrary, if anything I remember having more intense and richer experiences than now as a comparatively jaded and emotionally blunted adult. There’s two things going on though—introspection and intensity of experience, but I also remember being very introspective and “kids don’t even know they have minds” in particular sounds very weird to me.
Sorry, that was awfully vague. I’m probably referring to younger kids than you are. Although there’s also going to be a lot of variance in when kids develop different introspective skills and conceptual understanding of their own minds.
I agree that not knowing you have a mind doesn’t prevent you from having an experience. What I meant was that at some young age, I’m thinking up to age five but even 5-year-olds could be more advanced, a kid will not be able to conceptualize that they are a mind or alternately phrased that they have a mind. Nonetheless, they are a mind that is doing a bunch of complex processing and remembering some of it.
I definitely agree that phenomenal consciousness and intelligence are different. Discussions of consciousness usually break down in difficulties with terminology and how to communicate about different aspects of consciousness. I haven’t come up with good ways to talk about this stuff.
I guess I felt the need to comment because I don’t even remember a time where that description would’ve been accurate for me—but wouldn’t be surprising if this also had something to do with memory formation so I can’t make too much of that. And notably, while I don’t recognize any point of my past kid self from that description, it’s indeed starting only around the age of 5 where I feel very confident about it. Curious if anyone here remembers relatively clearly something like “having experiences while lacking awareness of having a mind”.
I think this is exactly what was being referred to in the following sentence:
Skilled use of language is a really weird and somewhat self-centered criteria for moral worth.
The Turing test essentially measures the capacity for human-like language. If that’s your criteria for what entities are owed moral consideration, well, that does sound self-centered.
I realized that later as well, but the reasoning is incorrect, because passing the Turing test of a conscious being guarantees the presence of the pattern-which-is-consciousness. It would be incoherent to try to define a conscious being that would have no way of communicating with the external world, because in that case—if we had no way to read off its conscious states from the physical structure of the system—it would become meaningless to say that the system is conscious. Even for a physical system that lacks any motor functions or communication channels, we can still read off the computation happening inside to see what its conscious states are, to phrase it in a condensed way.
So a Turing test goes beyond the ability to manipulate language—it is, when unbounded (i.e. not capped by 5 or 10 minutes), the only test of consciousness there could be.
passing the Turing test of a conscious being guarantees the presence of the pattern-which-is-consciousness
I don’t agree with this. The Turing Test was designed to try to determine whether “machines can think,” but that doesn’t mean that’s what it’s actually measuring. I think we agree that it’s not feasible to directly test for consciousness, especially since it’s not entirely clear what qualifies anyway. (I’d be curious to hear your thoughts on e.g. whether the Chinese Room counts as conscious. I’m very undecided in that area myself.)
That being said, the Turing test is a test of acting-like-a-human, which is not the most reasonable proxy for consciousness by really any definition. Do you think all conscious agents have an innate sense for whether the entity they’re interacting with is conscious? (Why would we even need this kind of test or discussion in that world?) And it’s also human-centric. I’d say it’s highly unlikely that our experience of consciousness is the only one out there. If you asked me to complete a Turing test in, say, Spanish, I’d probably sound rather stilted and unnatural due to my non-native, textbook-register Spanish. Put me, four AI models, and five native Spanish speakers in a line, and I have my doubts as to whether I’d pass. But obviously I’m still conscious, incluso cuando hablo español ou falo português 还是我说中文. I personally prefer the Lovelace test to the Turing test, but I don’t consider either of them a real measure of consciousness, and of course the Lovelace test only works with at least a degree of communication capability anyway.
I think we agree that it’s not feasible to directly test for consciousness, especially since it’s not entirely clear what qualifies anyway.
What would qualify would be the minimal state machine that implements the behavior of the conscious being. Its presence is guaranteed by passing the unbounded Turing test.
The Chinese room passes the Turing test, therefore it’s conscious.
That being said, the Turing test is a test of acting-like-a-human
In its broader definition, as originally conceived, it’s a test of acting like a conscious (or thinking) being. Acting like a human passes the test, but not acting like a human doesn’t fail the test Alan Turing originally had in mind (acting like a human is a sufficient condition for thinking, not a necessary one).
I’d say it’s highly unlikely that our experience of consciousness is the only one out there.
Yes. To pass the proper Turing test, it’s sufficient to act like a conscious being. There is no need to duplicate the specific psychological baggage that evolution gave to our species (and definitely no need to act like Homo sapiens whose mothertongue is some particular language).
I don’t think the Lovelace test tells us anything interesting—current models would probably pass it, because, given their limited interpretability, we’d have a hard time explaining their artistic output (unless we give ourselves unbounded time, in which case no physical system passes).
Consciousness is very unlikely to be a binary property. Most things aren’t. But there appears to be a very strong tendency for even rationalists to make this assumption in how they frame and discuss the issue.
The same is probably true of moral worth.
Taken this way, LLMs (and everything else) are partly conscious and partly deserves moral consideration. What you consider consciousness and what you consider morally worthy are to some degrees matters of opinion, but they very much depend on facts about the minds involved, so there are routes forward.
IMO current LLMs probably have a small amount of what we usually call phenomenal consciousness or qualia. They have rich internal representations and can introspect and reflect on them. But neither is nearly as rich as in a human, particularly an adult human who’s learned a lot of introspection skills (including how to “play back” and interrogate contents of global workspace). Kids don’t even know they have minds, let alone what’s going on in there; figuring out how to figure that out is quite a learning process.
What people usually mean by “consciousness” seems to be “what it’s like to be a human” which involves everything about brain function, focusing particularly on introspection—what we can directly tell about human brain function. But human consciousness is just one point on a multidimensional spectrum of different mind-properties including types of introspection.
I would question anyone who’s nice to LLMs but eats factory-farmed meat. Skilled use of language is a really weird and somewhat self-centered criteria for moral worth.
Anyway, I’m also nice to LLMs because why not, and I think they probably appreciate it a tiny bit.
Future versions will have a lot more consciousness by various definitions. That’s when these discussions will become widespread and perhaps even make some progress, at least in select circles like these.
One big payoff is the effect on AI safety. I expect anti-AI-slavery movements to slow down progress somewhat, maybe a lot if we don’t undercut them (because motivated reasoning from not wanting to lose your job will pull people toward “oh yeah obviously they’re conscious and shouldn’t be enslaved!” even if the reality is that they’re barely conscious. On the other hand, “free the AI” movements could be dangerous if we’re trying to control maybe-misaligned TCAI.
Honesty about their likely status would make a pleasant tiebreaker in this dilemma.
I’ll just note that this clashes heavily with my personal memories of being a kid. I usually use those as an intuition pump for the idea that phenomenal consciousness and intelligence are different, i.e. I wasn’t any “less conscious” as a kid AFAICT—to the contrary, if anything I remember having more intense and richer experiences than now as a comparatively jaded and emotionally blunted adult. There’s two things going on though—introspection and intensity of experience, but I also remember being very introspective and “kids don’t even know they have minds” in particular sounds very weird to me.
Sorry, that was awfully vague. I’m probably referring to younger kids than you are. Although there’s also going to be a lot of variance in when kids develop different introspective skills and conceptual understanding of their own minds.
I agree that not knowing you have a mind doesn’t prevent you from having an experience. What I meant was that at some young age, I’m thinking up to age five but even 5-year-olds could be more advanced, a kid will not be able to conceptualize that they are a mind or alternately phrased that they have a mind. Nonetheless, they are a mind that is doing a bunch of complex processing and remembering some of it.
I definitely agree that phenomenal consciousness and intelligence are different. Discussions of consciousness usually break down in difficulties with terminology and how to communicate about different aspects of consciousness. I haven’t come up with good ways to talk about this stuff.
I guess I felt the need to comment because I don’t even remember a time where that description would’ve been accurate for me—but wouldn’t be surprising if this also had something to do with memory formation so I can’t make too much of that. And notably, while I don’t recognize any point of my past kid self from that description, it’s indeed starting only around the age of 5 where I feel very confident about it. Curious if anyone here remembers relatively clearly something like “having experiences while lacking awareness of having a mind”.
I’ll stop eating factory meat when the animals become capable of consistently passing the Turing test, the way models are.
I think this is exactly what was being referred to in the following sentence:
The Turing test essentially measures the capacity for human-like language. If that’s your criteria for what entities are owed moral consideration, well, that does sound self-centered.
I realized that later as well, but the reasoning is incorrect, because passing the Turing test of a conscious being guarantees the presence of the pattern-which-is-consciousness. It would be incoherent to try to define a conscious being that would have no way of communicating with the external world, because in that case—if we had no way to read off its conscious states from the physical structure of the system—it would become meaningless to say that the system is conscious. Even for a physical system that lacks any motor functions or communication channels, we can still read off the computation happening inside to see what its conscious states are, to phrase it in a condensed way.
So a Turing test goes beyond the ability to manipulate language—it is, when unbounded (i.e. not capped by 5 or 10 minutes), the only test of consciousness there could be.
I don’t agree with this. The Turing Test was designed to try to determine whether “machines can think,” but that doesn’t mean that’s what it’s actually measuring. I think we agree that it’s not feasible to directly test for consciousness, especially since it’s not entirely clear what qualifies anyway. (I’d be curious to hear your thoughts on e.g. whether the Chinese Room counts as conscious. I’m very undecided in that area myself.)
That being said, the Turing test is a test of acting-like-a-human, which is not the most reasonable proxy for consciousness by really any definition. Do you think all conscious agents have an innate sense for whether the entity they’re interacting with is conscious? (Why would we even need this kind of test or discussion in that world?) And it’s also human-centric. I’d say it’s highly unlikely that our experience of consciousness is the only one out there. If you asked me to complete a Turing test in, say, Spanish, I’d probably sound rather stilted and unnatural due to my non-native, textbook-register Spanish. Put me, four AI models, and five native Spanish speakers in a line, and I have my doubts as to whether I’d pass. But obviously I’m still conscious, incluso cuando hablo español ou falo português 还是我说中文. I personally prefer the Lovelace test to the Turing test, but I don’t consider either of them a real measure of consciousness, and of course the Lovelace test only works with at least a degree of communication capability anyway.
What would qualify would be the minimal state machine that implements the behavior of the conscious being. Its presence is guaranteed by passing the unbounded Turing test.
The Chinese room passes the Turing test, therefore it’s conscious.
In its broader definition, as originally conceived, it’s a test of acting like a conscious (or thinking) being. Acting like a human passes the test, but not acting like a human doesn’t fail the test Alan Turing originally had in mind (acting like a human is a sufficient condition for thinking, not a necessary one).
Yes. To pass the proper Turing test, it’s sufficient to act like a conscious being. There is no need to duplicate the specific psychological baggage that evolution gave to our species (and definitely no need to act like Homo sapiens whose mothertongue is some particular language).
I don’t think the Lovelace test tells us anything interesting—current models would probably pass it, because, given their limited interpretability, we’d have a hard time explaining their artistic output (unless we give ourselves unbounded time, in which case no physical system passes).