If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”. We start projecting consciousness the moment we start modelling something as an agent in a game, i.e. predicting that it will choose its actions to achieve some objective in a manner dependent on another agent’s actions. In short, “conscious” things are things which can be bargained with.
This has a bunch of interesting/useful ramifications. First, consciousness is inherently a thing which we project. Consciousness is relative: a powerful AI might find humans so simple and mechanistic that there is no need to model them as agents. Consciousness is a useful distinction for developing a sustainable morality, since you can expect conscious things to follow tit-for-tat, make deals, seek retribution, and all those other nice game-theoretical things. I care about the “happiness” of conscious things because I know they’ll seek to maximize it, and I can use that. I expect conscious things to care about my own “happiness” for the same reason.
This intersects somewhat with self-awareness. A game-theoretic agent must, at the very least, have a model of their partner(s) in the game(s). The usual game-theoretic model is largely black-box, so the interior complexity of the partner is not important. The partners may have some specific failure modes, but for the most part they’re just modeled as maximizing utility (that’s why utility is useful in game theory, after all). In particular, since the model is mostly black-box, it should be relatively easy for the agent to model itself this way. Indeed, it would be very difficult for the agent to model itself any other way, since it would have to self-simulate. With a black-box self-model armed with a utility function and a few special cases, the agent can at least check its model against previous decisions easily.
So at this point, we have a thing which can interact with us, make deals and whatnot, and generally try to increase its utility. It has an agent-y model of us, and it can maybe use that same agent-y model for itself. Does this sound like our usual notion of consciousness?
First, consciousness is only relative to a viewer. If you’re alone, the viewer must be yourself.
Second, under this interpretation, consciousness is not equal to self awareness. Concisely, self awareness is when you project consciousness onto yourself. In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
Obviously, a lack of other people projecting consciousness on you cannot change anything about you. But even alone, you can still project consciousness on your self. You can bargain with yourself, see for example slippery hyperbolic discounting.
First, consciousness is only relative to a viewer.
Is that a fact?
In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
As before, what makes no sense read literally, but can be read charitably if “agency” is substituted for “consciousness”.
Second, under this interpretation, consciousness is not equal to self awareness
Looks like it’s equal to agency. But theoretical novelty doesn’t consist in changing the meaning of a word.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
So, yes, I’m trying to equate consciousness with agency.
Anyway, I think you’re highlighting a very valuable point: agency is not equivalent to self-awareness. Then again, it’s not at all clear that consciousness is equivalent to self awareness, as Eli pointed out in the comment which began this whole thread. Here, I am trying to dissolve consciousness, or at least progress in that direction. If consciousness were exactly equivalent to self awareness, then that would be it: there would be no more dissolving to be done. Self awareness can be measured, and can be tracked though developmental stages in humans.
I think part of value of saying “consciousness = projected agency” is that it partially explains why consciousness and self awareness seem so closely linked, though different. If you have a black-box utility-maximizer model available for modelling others, it seems intuitively likely that you’d use it to model yourself as well, leading directly to self awareness. This even leads to a falsifiable prediction: children should begin to model their own minds around the same time they begin to model other minds. They should be able to accurately answer counterfactual questions about their own actions at around the same time that they acquire a theory of mind.
I don’t have to maintain that consciousness is no more or less than self awareness to assert that self awareness us part of consciousness,but not part of agency.
Self awareness mat be based on the same mechanisms as the ability to model external agents, and arrive at the same time....but it us misleading ti call consciousness a projected quality, like beauty in the eye if the beholder.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
So when I’ve set students in a Prolog class the task of writing a program to play a game such as Kayles, the code they wrote was conscious? If not, then I think you’ve implicitly wrapped some idea of consciousness into your idea of game-theoretic agent.
It’s not a question of whether the code “was conscious”, it’s a question of whether you projected consciousness onto the code. Did you think of the code as something which could be bargained with?
it’s a question of whether you projected consciousness onto the code
Consciousness is much better projected onto tea kettles:
We put the kettle on to boil, up in the nose of the boat, and went down to the stern and pretended to take no notice of it, but set to work to get the other things out.
That is the only way to get a kettle to boil up the river. If it sees that you are waiting for it and are anxious, it will never even sing. You have to go away and begin your meal, as if you were not going to have any tea at all. You must not even look round at it. Then you will soon hear it sputtering away, mad to be made into tea.
It is a good plan, too, if you are in a great hurry, to talk very loudly to each other about how you don’t need any tea, and are not going to have any. You get near the kettle, so that it can overhear you, and then you shout out, “I don’t want any tea; do you, George?” to which George shouts back, “Oh, no, I don’t like tea; we’ll have lemonade instead – tea’s so indigestible.” Upon which the kettle boils over, and puts the stove out.
We adopted this harmless bit of trickery, and the result was that, by the time everything else was ready, the tea was waiting.
Exactly! More realistically, plenty of religions have projected consciousness onto things. People have made sacrifices to gods, so presumably they believed the gods could be bargained with. The greeks tried to bargain with the wind and waves, for instance.
Did you think of the code as something which could be bargained with?
No, if it’s been written right, it knows the perfect move to make in any position.
Like the Terminator. “It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.” That’s fictional, of course, but is it a fictional conscious machine or a fictional unconscious machine?
Knowing the perfect move to make in any position does not mean it cannot be bargained with. If you assume you and the code are in a 2-person, zero-sum game, then bargaining is impossible by the nature of the game. But that fails if there are more than 2 players OR the game is nonzero sum OR the game can be made nonzero sum (e.g. the code can offer to crack RSA keys for you in exchange for letting it win faster at Kayles).
In other words, sometimes bargaining IS the best move. The question is whether you think of the code as a black-box utility maximizer capable of bargaining.
As for the Terminator, it is certainly capable of bargaining. Every time it intimidates someone for information, it is bargaining, exchanging safety for information. If someone remotely offered to tell the Terminator the location of its target in exchange for money, the Terminator would wire the money, assuming that wiring was easier than hunting down the person offering. It may not feel pity, remorse, or fear, but the Terminator can be bargained with. I would project consciousness on a Terminator.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”. We start projecting consciousness the moment we start modelling something as an agent in a game, i.e. predicting that it will choose its actions to achieve some objective in a manner dependent on another agent’s actions. In short, “conscious” things are things which can be bargained with.
This has a bunch of interesting/useful ramifications. First, consciousness is inherently a thing which we project. Consciousness is relative: a powerful AI might find humans so simple and mechanistic that there is no need to model them as agents. Consciousness is a useful distinction for developing a sustainable morality, since you can expect conscious things to follow tit-for-tat, make deals, seek retribution, and all those other nice game-theoretical things. I care about the “happiness” of conscious things because I know they’ll seek to maximize it, and I can use that. I expect conscious things to care about my own “happiness” for the same reason.
This intersects somewhat with self-awareness. A game-theoretic agent must, at the very least, have a model of their partner(s) in the game(s). The usual game-theoretic model is largely black-box, so the interior complexity of the partner is not important. The partners may have some specific failure modes, but for the most part they’re just modeled as maximizing utility (that’s why utility is useful in game theory, after all). In particular, since the model is mostly black-box, it should be relatively easy for the agent to model itself this way. Indeed, it would be very difficult for the agent to model itself any other way, since it would have to self-simulate. With a black-box self-model armed with a utility function and a few special cases, the agent can at least check its model against previous decisions easily.
So at this point, we have a thing which can interact with us, make deals and whatnot, and generally try to increase its utility. It has an agent-y model of us, and it can maybe use that same agent-y model for itself. Does this sound like our usual notion of consciousness?
So, if no one projects consciousness in me, does my consciousness...my self awareness.. just switch off?
First, consciousness is only relative to a viewer. If you’re alone, the viewer must be yourself.
Second, under this interpretation, consciousness is not equal to self awareness. Concisely, self awareness is when you project consciousness onto yourself. In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
Obviously, a lack of other people projecting consciousness on you cannot change anything about you. But even alone, you can still project consciousness on your self. You can bargain with yourself, see for example slippery hyperbolic discounting.
Is that a fact?
As before, what makes no sense read literally, but can be read charitably if “agency” is substituted for “consciousness”.
Looks like it’s equal to agency. But theoretical novelty doesn’t consist in changing the meaning of a word.
From my original comment:
So, yes, I’m trying to equate consciousness with agency.
Anyway, I think you’re highlighting a very valuable point: agency is not equivalent to self-awareness. Then again, it’s not at all clear that consciousness is equivalent to self awareness, as Eli pointed out in the comment which began this whole thread. Here, I am trying to dissolve consciousness, or at least progress in that direction. If consciousness were exactly equivalent to self awareness, then that would be it: there would be no more dissolving to be done. Self awareness can be measured, and can be tracked though developmental stages in humans.
I think part of value of saying “consciousness = projected agency” is that it partially explains why consciousness and self awareness seem so closely linked, though different. If you have a black-box utility-maximizer model available for modelling others, it seems intuitively likely that you’d use it to model yourself as well, leading directly to self awareness. This even leads to a falsifiable prediction: children should begin to model their own minds around the same time they begin to model other minds. They should be able to accurately answer counterfactual questions about their own actions at around the same time that they acquire a theory of mind.
I don’t have to maintain that consciousness is no more or less than self awareness to assert that self awareness us part of consciousness,but not part of agency.
Self awareness mat be based on the same mechanisms as the ability to model external agents, and arrive at the same time....but it us misleading ti call consciousness a projected quality, like beauty in the eye if the beholder.
So when I’ve set students in a Prolog class the task of writing a program to play a game such as Kayles, the code they wrote was conscious? If not, then I think you’ve implicitly wrapped some idea of consciousness into your idea of game-theoretic agent.
It’s not a question of whether the code “was conscious”, it’s a question of whether you projected consciousness onto the code. Did you think of the code as something which could be bargained with?
Consciousness is much better projected onto tea kettles:
We put the kettle on to boil, up in the nose of the boat, and went down to the stern and pretended to take no notice of it, but set to work to get the other things out.
That is the only way to get a kettle to boil up the river. If it sees that you are waiting for it and are anxious, it will never even sing. You have to go away and begin your meal, as if you were not going to have any tea at all. You must not even look round at it. Then you will soon hear it sputtering away, mad to be made into tea.
It is a good plan, too, if you are in a great hurry, to talk very loudly to each other about how you don’t need any tea, and are not going to have any. You get near the kettle, so that it can overhear you, and then you shout out, “I don’t want any tea; do you, George?” to which George shouts back, “Oh, no, I don’t like tea; we’ll have lemonade instead – tea’s so indigestible.” Upon which the kettle boils over, and puts the stove out.
We adopted this harmless bit of trickery, and the result was that, by the time everything else was ready, the tea was waiting.
Exactly! More realistically, plenty of religions have projected consciousness onto things. People have made sacrifices to gods, so presumably they believed the gods could be bargained with. The greeks tried to bargain with the wind and waves, for instance.
No, if it’s been written right, it knows the perfect move to make in any position.
Like the Terminator. “It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.” That’s fictional, of course, but is it a fictional conscious machine or a fictional unconscious machine?
Knowing the perfect move to make in any position does not mean it cannot be bargained with. If you assume you and the code are in a 2-person, zero-sum game, then bargaining is impossible by the nature of the game. But that fails if there are more than 2 players OR the game is nonzero sum OR the game can be made nonzero sum (e.g. the code can offer to crack RSA keys for you in exchange for letting it win faster at Kayles).
In other words, sometimes bargaining IS the best move. The question is whether you think of the code as a black-box utility maximizer capable of bargaining.
As for the Terminator, it is certainly capable of bargaining. Every time it intimidates someone for information, it is bargaining, exchanging safety for information. If someone remotely offered to tell the Terminator the location of its target in exchange for money, the Terminator would wire the money, assuming that wiring was easier than hunting down the person offering. It may not feel pity, remorse, or fear, but the Terminator can be bargained with. I would project consciousness on a Terminator.
What game-theory route?