It seems that consciousness is one of the essential things necessary to characterize the moral status of an agent. It seems that we have very little chance to solve the AI safety problem. We don’t know if the first AGI will be conscious.
Since we are in a certain sense doomed, the future consists of the following 2 cases:
Either the first AGI is not sentient and never becomes sentient.
Or the first AGI acquires sentience at a given moment.
Either the valence of this AGI is positive
Or the valence of this AGI is negative
Either the valence is mixed
Or the AGI reprograms itself to experience a positive valence (instrumental convergence if its well-being is included in the utility function?)
I have no idea which scenario is the most desirable. The uncertainty is multiplied by considering the AGI’s superior capabilities, and its ability to multiply. So perhaps we could see it as a utility monster. Therefore, the following questions seem very important:
(1) What do you think is the most likely scenario and (2) the most desirable scenario?
(3|2) Conditional on the scenario that seems most desirable to you, is there a way to steer the future in that direction?
I agree that today the concept of consciousness is very poorly defined, but I think that in the future it will be possible to define it in a way that will make sense, or at least that we will be able to correct our current intuitions.
How can one tell if a human is conscious?
In humans, we have clues. For example, it is possible to experiment by varying the duration of a light stimulus. There are stimuli of very short duration that are processed by visual areas V1 and V2, but which do not go up to the parietal cortex. For stimuli of slightly longer duration, the light spot induces a conscious response. The subject can then say: yes, I saw the spot. In humans, the study can be done on a declarative basis.
How can one tell if an AI is conscious?
At present, such an experiment cannot be performed on GPT (unless an analogy is made between the duration of a stimulus for humans and the activation in the attention layers of the transformer?)
Indications of consciousness in an AI would be: - A global workspace, which allows a central piece of information to be accessible from anywhere in the system (e.g. in the schema of the paper attention is all you need, the link between the last encoder and all decoders, or recently in the Socratic models, an implementation of a common language module) - consciousness seems to be a way of processing information in a complete Turing fashion, but it is quite slow, whereas unconscious processes automatically a multitude of bits in parallel. So I would guess another clue would be a distinction between several modes of functioning:
an automatic mode, system 1, such as GPU parallelized vision networks today
vs a slower mode, system 2, which would be a very general program which cannot be parallelized and should be run on cpu.
- Perhaps an implementation of metacognition or a reflexive system: parts of the neural network that attempt to predict the future or current state of other parts of the neural network. But this point depends on whether one considers the mirror test to be necessary for consciousness.
Yeah, I know, I just wanted to begin answering with this and to present in one sentence (and without mentioning it, my bad...) the concept of neural correlates of consciousness
Are you using “conscious” and “sentient” as synonyms? Do you think some, most, or all humans are conscious? Why do you think this?
I’m not sure what it means for it to be “desirable” for an entity to be conscious. From a utilitarian standpoint, there’s some controversy on how to account for size of population even in humans or animals, but it’s generally accepted that if something IS conscious, then it’s better for that thing to have more positive experiences.
Are you using “conscious” and “sentient” as synonyms? → Yes. Maybe sentient is consciousness with only valence but it is basically the same thing. But you’re right, I shouldn’t have used both worlds at the same time without further explanation.
Do you think some, most, or all humans are conscious? → Yes, I think most humans are conscious. Why do you think this? Because I am conscious, but I’m not sure if I really answer your question here.
“I’m not sure what it means for it to be “desirable” for an entity to be conscious.” → Yes this is a good question!
Is it desirable for a stone to be conscious? Um, yeah. I’m not sure what that means.
Is it desirable for a human to be visually conscious? Yes. I don’t want you to become Blindsight.
Is it desirable for a human to be conscious? By extension, yes, I think so.
[Question] Is it desirable for the first AGI to be conscious?
It seems that consciousness is one of the essential things necessary to characterize the moral status of an agent.
It seems that we have very little chance to solve the AI safety problem.
We don’t know if the first AGI will be conscious.
Since we are in a certain sense doomed, the future consists of the following 2 cases:
Either the first AGI is not sentient and never becomes sentient.
Or the first AGI acquires sentience at a given moment.
Either the valence of this AGI is positive
Or the valence of this AGI is negative
Either the valence is mixed
Or the AGI reprograms itself to experience a positive valence (instrumental convergence if its well-being is included in the utility function?)
I have no idea which scenario is the most desirable. The uncertainty is multiplied by considering the AGI’s superior capabilities, and its ability to multiply. So perhaps we could see it as a utility monster. Therefore, the following questions seem very important:
(1) What do you think is the most likely scenario and (2) the most desirable scenario?
(3|2) Conditional on the scenario that seems most desirable to you, is there a way to steer the future in that direction?
How can one tell if an AI is conscious/sentient/sapient?
I agree that today the concept of consciousness is very poorly defined, but I think that in the future it will be possible to define it in a way that will make sense, or at least that we will be able to correct our current intuitions.
How can one tell if a human is conscious?
In humans, we have clues. For example, it is possible to experiment by varying the duration of a light stimulus. There are stimuli of very short duration that are processed by visual areas V1 and V2, but which do not go up to the parietal cortex. For stimuli of slightly longer duration, the light spot induces a conscious response. The subject can then say: yes, I saw the spot. In humans, the study can be done on a declarative basis.
How can one tell if an AI is conscious?
At present, such an experiment cannot be performed on GPT (unless an analogy is made between the duration of a stimulus for humans and the activation in the attention layers of the transformer?)
Indications of consciousness in an AI would be:
- A global workspace, which allows a central piece of information to be accessible from anywhere in the system (e.g. in the schema of the paper attention is all you need, the link between the last encoder and all decoders, or recently in the Socratic models, an implementation of a common language module)
- consciousness seems to be a way of processing information in a complete Turing fashion, but it is quite slow, whereas unconscious processes automatically a multitude of bits in parallel. So I would guess another clue would be a distinction between several modes of functioning:
an automatic mode, system 1, such as GPU parallelized vision networks today
vs a slower mode, system 2, which would be a very general program which cannot be parallelized and should be run on cpu.
- Perhaps an implementation of metacognition or a reflexive system: parts of the neural network that attempt to predict the future or current state of other parts of the neural network. But this point depends on whether one considers the mirror test to be necessary for consciousness.
- ?
That’s the easy case ..each of us we can tell we are conscious by introspection, and a normal-seeming and behaving person isn’t that different.
Yeah, I know, I just wanted to begin answering with this and to present in one sentence (and without mentioning it, my bad...) the concept of neural correlates of consciousness
Are you using “conscious” and “sentient” as synonyms? Do you think some, most, or all humans are conscious? Why do you think this?
I’m not sure what it means for it to be “desirable” for an entity to be conscious. From a utilitarian standpoint, there’s some controversy on how to account for size of population even in humans or animals, but it’s generally accepted that if something IS conscious, then it’s better for that thing to have more positive experiences.
Are you using “sentient” to mean “has senses” , “has qualia” , or “is a moral patient”?
Are you using “conscious” and “sentient” as synonyms? → Yes. Maybe sentient is consciousness with only valence but it is basically the same thing. But you’re right, I shouldn’t have used both worlds at the same time without further explanation.
Do you think some, most, or all humans are conscious? → Yes, I think most humans are conscious. Why do you think this? Because I am conscious, but I’m not sure if I really answer your question here.
“I’m not sure what it means for it to be “desirable” for an entity to be conscious.” → Yes this is a good question!
Is it desirable for a stone to be conscious? Um, yeah. I’m not sure what that means.
Is it desirable for a human to be visually conscious? Yes. I don’t want you to become Blindsight.
Is it desirable for a human to be conscious? By extension, yes, I think so.