“The stakes are very high for this ‘guess’. The ethical implications of getting it wrong are huge.”
True.
“The designers of the simulation or emulation fully intend to pass the Turing test; that is, it is the explicit purpose of the designers of the software to fool the interviewer.”
To clarify, I’m talking about something like a Moravec transfer, not a chatbot. Maybe a really sophisticated chatbot could pass the Turing test, but if we know that a given program was designed simply to game the Turing test, then we won’t be impressed by its passing the test. The designers aren’t trying to fool the interviewer; they’re trying to build a brain (or something that does the same kind of thing). We know that brains exist.
“I don’t see why the burden of proof should be on me.”
The reason is that the human brain is not magic. It’s doing something, and whatever that something is, it would incredibly surprising if it’s the only structure in the vastness of the space of all possible things that could do it. Yes, consciousness is a mystery unto me, and I’m waving my hands. I don’t know how to build a person. But the burden of proof is still on you.
To clarify, I’m talking about something like a Moravec transfer, not a chatbot. Maybe a really sophisticated chatbot could pass the Turing test, but if we know that a given program was designed simply to game the Turing test, then we won’t be impressed by its passing the test. The designers aren’t trying to fool the interviewer; they’re trying to build a brain (or something that does the same kind of thing). We know that brains exist.
“I don’t see why the burden of proof should be on me.”
The reason is that the human brain is not magic. It’s doing something, and whatever that something is, it would incredibly surprising if it’s the only structure in the vastness of the space of all possible things that could do it. Yes, consciousness is a mystery unto me, and I’m waving my hands. I don’t know how to build a person. But the burden of proof is still on you.