What if the person claims to be able to add numbers? If you ask them about 2+2 and they answer 4, maybe they were pre-ordered with that response, but if you get them to add a few dozen poisson-distributed numbers, maybe you start believing they’re actually implementing the algorithm. This relies on the important distinction between telling two things apart certainly and gathering evidence.
Unlike with addition, I don’t think we understand consciousness well enough to create a sequence of questions such that the simplest algorithm answering them would be conscious. It’s not clear to me that such a sequence even exists. If we found one, it would be a big step for FAI.
Hm, this is an interesting question to think about. I lean more towards the camp of construing consciousness as broad and pretty easy to attain, but only a small part of a mind’s value. As long as we can push down the probability of lookup tables and push up the probability of self-reflection and abstract thinking.
Weird example I’d label as conscious: an AI that can observe us, trying to fool us in a particular way: Our brains compute expectations of what kinds of things a conscious correspondent would say, then the AI can observe these expectations and compute something consistent both with our expectations and its past responses. Most of the computation of a mind is there, but packaged differently and spread over multiple media—the text responses no longer reflect consciousness if the AI loses its observation channel.
What if the person claims to be able to add numbers? If you ask them about 2+2 and they answer 4, maybe they were pre-ordered with that response, but if you get them to add a few dozen poisson-distributed numbers, maybe you start believing they’re actually implementing the algorithm. This relies on the important distinction between telling two things apart certainly and gathering evidence.
Unlike with addition, I don’t think we understand consciousness well enough to create a sequence of questions such that the simplest algorithm answering them would be conscious. It’s not clear to me that such a sequence even exists. If we found one, it would be a big step for FAI.
Hm, this is an interesting question to think about. I lean more towards the camp of construing consciousness as broad and pretty easy to attain, but only a small part of a mind’s value. As long as we can push down the probability of lookup tables and push up the probability of self-reflection and abstract thinking.
Weird example I’d label as conscious: an AI that can observe us, trying to fool us in a particular way: Our brains compute expectations of what kinds of things a conscious correspondent would say, then the AI can observe these expectations and compute something consistent both with our expectations and its past responses. Most of the computation of a mind is there, but packaged differently and spread over multiple media—the text responses no longer reflect consciousness if the AI loses its observation channel.