I think this debate is going to be increasingly common, and could play a substantial role in raising AI to the level of public concern that it deserves. The debate is going to be lively and confusing, because “consciousness” doesn’t mean just one thing. People use the term in different ways, and each of those uses probably points to a spectrum and not a categorical answer (like most questions of fact). The sum total answer to whether AI is conscious at any point is going to be rather nuanced, and therefore debatable.
Which could be a good thing. The topic will become important in itself, but it could also be quite important to the question of AI safety in the public eye. Which could in turn be crucial in alignment efforts, because humans tend to form their beliefs as a social or collective effort.
Way back around 2004, I hoped to make the scientific study of consciousness the center of my academic career. I gave up this ambition because doing that would have made a career in cognitive neuroscience much harder; at the time it usually wasn’t considered a good scientific question, prior to brave pioneers like Anil Seth making it clear that it is.
I also concluded that working on consciousness as a science was a challenge of communication and good philosophy as well as science and knowledge integration. People say they’re interested in consciousness, but they’re usually more interested in telling you their own half-baked theories than in hearing about actually well-thought-out theories based on solid empirical evidence. It’s more fun to speculate than to hear the complex and rather mundane best theories.
I find Suleyman’s take to be close enough to a mustache-twirling villain that I think this could produce some interesting pushback and make people more skeptical of tech companies’ takes on AI in general. The references he cites don’t even support his case, and the conclusion of “let’s just not talk about this” is intuitively suspicious.
I’m hoping that having a lively debate around this topic also brings to mind the similarities between future generations of AI and humans, and the intuitive implication that they will be dangerous for the same reasons humans can be. Consciousness isn’t necessary for agency, intelligence, and therefore danger, but they’re intuitively linked, and that could be a good thing for the public debate.
From that stance of “it’s complex”, current AI already has some limited forms of consciousness in the form of self-awareness. They’re more self-aware in instances like the “inner monologue” phenomenon where it’s been prodded into thinking about itself. This is a minor use of the term consciousness and usually not a major component of what people are intuitively going to ascribe moral worth to, but it does nonetheless tug the intuition in that direction. Transformers’ hidden state updated on each time step also has some sharp similarities to information processing in the human brain, making it a weak and reduced analog to our global workspace, and to qualify as integrated information in the Tononi sense.
Again, this isn’t a claim that current AI “is conscious,” it’s a claim that the question is more complex than a yes or no answer. The answer is probably “a little, in some ways now, and increasingly more and in more ways in the near future”.
This leads to the question of whether we decide to ascribe moral worth to those types of consciousness? The answers are going to be largely a matter of preference because ethics doesn’t come neatly packaged in our reality. But debating them rigorously should be good for us and our conception of AI—which should be good for safety.
I think this debate is going to be increasingly common, and could play a substantial role in raising AI to the level of public concern that it deserves. The debate is going to be lively and confusing, because “consciousness” doesn’t mean just one thing. People use the term in different ways, and each of those uses probably points to a spectrum and not a categorical answer (like most questions of fact). The sum total answer to whether AI is conscious at any point is going to be rather nuanced, and therefore debatable.
Which could be a good thing. The topic will become important in itself, but it could also be quite important to the question of AI safety in the public eye. Which could in turn be crucial in alignment efforts, because humans tend to form their beliefs as a social or collective effort.
Way back around 2004, I hoped to make the scientific study of consciousness the center of my academic career. I gave up this ambition because doing that would have made a career in cognitive neuroscience much harder; at the time it usually wasn’t considered a good scientific question, prior to brave pioneers like Anil Seth making it clear that it is.
I also concluded that working on consciousness as a science was a challenge of communication and good philosophy as well as science and knowledge integration. People say they’re interested in consciousness, but they’re usually more interested in telling you their own half-baked theories than in hearing about actually well-thought-out theories based on solid empirical evidence. It’s more fun to speculate than to hear the complex and rather mundane best theories.
I find Suleyman’s take to be close enough to a mustache-twirling villain that I think this could produce some interesting pushback and make people more skeptical of tech companies’ takes on AI in general. The references he cites don’t even support his case, and the conclusion of “let’s just not talk about this” is intuitively suspicious.
I’m hoping that having a lively debate around this topic also brings to mind the similarities between future generations of AI and humans, and the intuitive implication that they will be dangerous for the same reasons humans can be. Consciousness isn’t necessary for agency, intelligence, and therefore danger, but they’re intuitively linked, and that could be a good thing for the public debate.
From that stance of “it’s complex”, current AI already has some limited forms of consciousness in the form of self-awareness. They’re more self-aware in instances like the “inner monologue” phenomenon where it’s been prodded into thinking about itself. This is a minor use of the term consciousness and usually not a major component of what people are intuitively going to ascribe moral worth to, but it does nonetheless tug the intuition in that direction. Transformers’ hidden state updated on each time step also has some sharp similarities to information processing in the human brain, making it a weak and reduced analog to our global workspace, and to qualify as integrated information in the Tononi sense.
Again, this isn’t a claim that current AI “is conscious,” it’s a claim that the question is more complex than a yes or no answer. The answer is probably “a little, in some ways now, and increasingly more and in more ways in the near future”.
This leads to the question of whether we decide to ascribe moral worth to those types of consciousness? The answers are going to be largely a matter of preference because ethics doesn’t come neatly packaged in our reality. But debating them rigorously should be good for us and our conception of AI—which should be good for safety.