My take on p-zombies, AI consciousness, and all related questions: sure, whatever. Anything that claims to be conscious is probably conscious. Qualia are a side effect of at least some kinds of compute, and there’s not much reason to believe it’s ONLY biological, and almost no reason to believe it’s divorced from the capabilities.
I don’t think consciousness is always similar to my own experiences, especially in terms of valence, tropism (pain/pleasure), or emotional inner experience (though I do recognize external similarity in behavior among other people, it could be a very different experience causing it).
Where I disagree with most is that my view of morality doesn’t demand that I give equal weight in my utility function to all things that are conscious. There are plenty of conscious beings that I care about orders of magnitude less than similarly-conscious beings closer to me. Pigs and cows are likely conscious, and I’ll still support farming them for food. AI is near-conscious (with MASSIVE differences in terms of continuity and dimensionality), and I still will ask it to do my coding tasks and answer dumb questions on the internet.
Anything that claims to be conscious is probably conscious.
This glosses over roughly all of the interesting cases—does this apply to a python program that always prints “I am conscious”? If not, then does it apply to an LLM that was trained to output similar text? Do you think the reverse is also true—that most things that don’t claim to be conscious is probably not conscious? (I suppose no since you grant animals consciousness)
There are plenty of conscious beings that I care about orders of magnitude less than similarly-conscious beings closer to me.
You can do that but then it is roughly impossible to make moral progress, or have useful debates with people who don’t already share your intuitions. In order for that to work, I think you need to appeal to some meta ethics principle. Utilitarianism appeals to a simplicity prior and a veil of ignorance kind of thing. Is there anything upstream of your assessment of moral patienthood?
This glosses over roughly all of the interesting cases—does this apply to a python program that always prints “I am conscious”? If not, then does it apply to an LLM that was trained to output similar text? Do you think the reverse is also true—that most things that don’t claim to be conscious is probably not conscious? (I suppose no since you grant animals consciousness)
I don’t find the python case interesting, but you’re right—I should amend to “things which I find plausibly conscious are probably actually conscious”. It needs to be complex enough that there are causal physical pathways which I can’t decompose to a simple dismiss-able model. And it doesn’t need to make the claim directly—other behaviors that I can infer consciousness from qualify.
does it apply to an LLM that was trained to output similar text?
It applies to human brains that have learned/been trained to generate those outputs. Why wouldn’t it apply to other complicated general-purpose predictors?
it is roughly impossible to make moral progress, or have useful debates with people who don’t already share your intuitions
Yes, I bite that bullet. Moral realism is misguided and incorrect. “moral progress” is refinement and update of my beliefs to make me more satisfied, and is discussion and alignment with other actors to come into agreement with my preferences (or optimal trade/compromise between our unresolved and unresolvable differences).
At the root, it’s all physics, including my and your preferences about what we’d like to perceive/experience.
Consciousness isn’t special.
My take on p-zombies, AI consciousness, and all related questions: sure, whatever. Anything that claims to be conscious is probably conscious. Qualia are a side effect of at least some kinds of compute, and there’s not much reason to believe it’s ONLY biological, and almost no reason to believe it’s divorced from the capabilities.
I don’t think consciousness is always similar to my own experiences, especially in terms of valence, tropism (pain/pleasure), or emotional inner experience (though I do recognize external similarity in behavior among other people, it could be a very different experience causing it).
Where I disagree with most is that my view of morality doesn’t demand that I give equal weight in my utility function to all things that are conscious. There are plenty of conscious beings that I care about orders of magnitude less than similarly-conscious beings closer to me. Pigs and cows are likely conscious, and I’ll still support farming them for food. AI is near-conscious (with MASSIVE differences in terms of continuity and dimensionality), and I still will ask it to do my coding tasks and answer dumb questions on the internet.
This glosses over roughly all of the interesting cases—does this apply to a python program that always prints “I am conscious”? If not, then does it apply to an LLM that was trained to output similar text? Do you think the reverse is also true—that most things that don’t claim to be conscious is probably not conscious? (I suppose no since you grant animals consciousness)
You can do that but then it is roughly impossible to make moral progress, or have useful debates with people who don’t already share your intuitions. In order for that to work, I think you need to appeal to some meta ethics principle. Utilitarianism appeals to a simplicity prior and a veil of ignorance kind of thing. Is there anything upstream of your assessment of moral patienthood?
I don’t find the python case interesting, but you’re right—I should amend to “things which I find plausibly conscious are probably actually conscious”. It needs to be complex enough that there are causal physical pathways which I can’t decompose to a simple dismiss-able model. And it doesn’t need to make the claim directly—other behaviors that I can infer consciousness from qualify.
It applies to human brains that have learned/been trained to generate those outputs. Why wouldn’t it apply to other complicated general-purpose predictors?
Yes, I bite that bullet. Moral realism is misguided and incorrect. “moral progress” is refinement and update of my beliefs to make me more satisfied, and is discussion and alignment with other actors to come into agreement with my preferences (or optimal trade/compromise between our unresolved and unresolvable differences).
At the root, it’s all physics, including my and your preferences about what we’d like to perceive/experience.
I think it would be more effective to consider the mental state of whoever wrote that program, not the program itself.