Hm, this is an interesting question to think about. I lean more towards the camp of construing consciousness as broad and pretty easy to attain, but only a small part of a mind’s value. As long as we can push down the probability of lookup tables and push up the probability of self-reflection and abstract thinking.
Weird example I’d label as conscious: an AI that can observe us, trying to fool us in a particular way: Our brains compute expectations of what kinds of things a conscious correspondent would say, then the AI can observe these expectations and compute something consistent both with our expectations and its past responses. Most of the computation of a mind is there, but packaged differently and spread over multiple media—the text responses no longer reflect consciousness if the AI loses its observation channel.
Hm, this is an interesting question to think about. I lean more towards the camp of construing consciousness as broad and pretty easy to attain, but only a small part of a mind’s value. As long as we can push down the probability of lookup tables and push up the probability of self-reflection and abstract thinking.
Weird example I’d label as conscious: an AI that can observe us, trying to fool us in a particular way: Our brains compute expectations of what kinds of things a conscious correspondent would say, then the AI can observe these expectations and compute something consistent both with our expectations and its past responses. Most of the computation of a mind is there, but packaged differently and spread over multiple media—the text responses no longer reflect consciousness if the AI loses its observation channel.