I agree with Richard K’s point here. I personally found H. Beam Piper’s sci fi novels on ‘Fuzzies’ to be a really good exploration of the boundaries of consciousness, sentience, and moral worth. Beam makes the distinction between ‘sentience’ as having animal awareness of self & environment and non-reflective consciousness, versus ‘sapience’ which involves a reflective self-awareness and abstract reasoning and thoughts about future and past and at least some sense of right and wrong.
So in this sense, I would call a cow conscious and sentient, but not sapient. I would call a honeybee sentient, capable of experiencing valenced experiences like pain or reward, but lacking in sufficient world- and self- modelling to be called conscious.
Personally, I wouldn’t say that a cow has no moral worth and it is fine to torture it. I do think that if you give a cow a good life, and then kill it in a quick mostly painless way, then that’s pretty ok. I don’t think that that’s ok to do to a human.
Philosophical reasoning about morality that doesn’t fall apart in edge cases or novel situations (e.g. sapient AI) is hard [citation needed]. My current guess, which I am not at all sure of, is that my morality says something about a qualitative difference between the moral value of sapient beings vs the moral value of non-sapient but conscious sentient beings vs non-sapient non-conscious sentient beings. To me, it seems no number of cow lives trades off against a human life, but cow QUALYs and dog QUALYs do trade off against each other at some ratio. Similarly, no number of non-conscious sentient lives like ants or worms trade off against a conscious and sapient life like a cow’s. I would not torture a single cow to save a billion shrimp from being tortured. Nor any number of shrimp. The value of the two seem non-commutative to me.
Are current language models or the entities they temporarily simulate sapient? I think not yet, but I do worry that at some point they will be. I think that as soon as this is the case, we have a strong moral obligation to avoid creating them, and if we do create them, to try to make sure they are treated ethically.
By my definitions are our LLMs or their simulated entities conscious? are they sentient? I’m unsure, but since I rank consciousness and sentience as of lower importance, I’m not too worried about the answers to these questions from a moral standpoint. Still fascinated from a scientific standpoint, of course.
Also, I think that there’s an even lower category than sentient. The example I like to use for this is a thermostat. It is agentic in that it is a system that responds behaviorally to changes in the environment (I’d call this a reflex perhaps, or stimulus/response pair), but it is not sentient because unlike a worm it doesn’t have a computational system that attaches valence to these reflexes. I think that there are entities which I would classify as living beings that fall into the non-sentient category. For example: I think probably coral polyps and maybe jellyfish have computational systems too simplistic for valence and thus respond purely reflexively. If this is the case, then I would not torture a single worm to save any number of coral polyps. I think most (non ML) computer programs fall into this category. I think a reinforcement learning agent transcends this category, by having valenced reactions to stimuli, and thus should be considered at least comparable to sentient beings like insects.
I like the distinctions you make between sentient, sapient, and conscious. I would like to bring up some thoughts about how to choose a morality that I think are relevant to your points about death of cows and transient beings, which I disagree with.
I think that when choosing our morality, we should do so under the assumption that we have been given complete omnipotent control over reality and that we should analyze all of our values independently, not taking into consideration any trade-offs, even when some of our values are logically impossible to satisfy simultaneously. Only after doing this do we start talking about what’s actually physically and logically possible and what trade-offs we are willing to make, while always making sure to be clear when something is actually part of our morality vs when something is a trade-off.
The reason for this approach is to avoid accidentally locking in trade-offs into our morality which might later turn out to not actually be necessary. And the great thing about it is that if we have not accidentally locked in any trade-offs into our morality, this approach should give back the exact same morality that we started off with, so when it doesn’t return the same answer I find it pretty instructive.
I think this applies to the idea that it’s okay to kill cows, because when I consider a world where I have to decide whether or not cows die, and this decision will not affect anything else in any way, then my intuition is that I slightly prefer that they not die. Therefore my morality is that cows should not die, even though in practice I think I might make similar trade-offs as you when it comes to cows in the world of today.
Something similar applies to transient computational subprocesses. If you had unlimited power and you had to explicitly choose if the things you currently call “transient computational subprocesses” are terminated, and you were certain that this choice would not affect anything else in any way at all (not even the things you think it’s logically impossible for it not to affect), would you still choose to terminate them? Remember that no matter what you choose here, you can still choose to trade things off the same way afterwards, so your answer doesn’t have to change your behavior in any way.
It’s possible that you still give the exact same answers with this approach, but I figure there’s a chance this might be helpful.
That’s an interesting way of reframing the issue. I’m honestly just not sure about all of this reasoning, and remain so after trying to think about it with your reframing, but I feel like this does shift my thinking a bit. Thanks.
I think probably it makes sense to try reasoning both with and without tradeoffs, and then comparing the results.
I agree with Richard K’s point here. I personally found H. Beam Piper’s sci fi novels on ‘Fuzzies’ to be a really good exploration of the boundaries of consciousness, sentience, and moral worth. Beam makes the distinction between ‘sentience’ as having animal awareness of self & environment and non-reflective consciousness, versus ‘sapience’ which involves a reflective self-awareness and abstract reasoning and thoughts about future and past and at least some sense of right and wrong.
So in this sense, I would call a cow conscious and sentient, but not sapient. I would call a honeybee sentient, capable of experiencing valenced experiences like pain or reward, but lacking in sufficient world- and self- modelling to be called conscious.
Personally, I wouldn’t say that a cow has no moral worth and it is fine to torture it. I do think that if you give a cow a good life, and then kill it in a quick mostly painless way, then that’s pretty ok. I don’t think that that’s ok to do to a human.
Philosophical reasoning about morality that doesn’t fall apart in edge cases or novel situations (e.g. sapient AI) is hard [citation needed]. My current guess, which I am not at all sure of, is that my morality says something about a qualitative difference between the moral value of sapient beings vs the moral value of non-sapient but conscious sentient beings vs non-sapient non-conscious sentient beings. To me, it seems no number of cow lives trades off against a human life, but cow QUALYs and dog QUALYs do trade off against each other at some ratio. Similarly, no number of non-conscious sentient lives like ants or worms trade off against a conscious and sapient life like a cow’s. I would not torture a single cow to save a billion shrimp from being tortured. Nor any number of shrimp. The value of the two seem non-commutative to me.
Are current language models or the entities they temporarily simulate sapient? I think not yet, but I do worry that at some point they will be. I think that as soon as this is the case, we have a strong moral obligation to avoid creating them, and if we do create them, to try to make sure they are treated ethically.
By my definitions are our LLMs or their simulated entities conscious? are they sentient? I’m unsure, but since I rank consciousness and sentience as of lower importance, I’m not too worried about the answers to these questions from a moral standpoint. Still fascinated from a scientific standpoint, of course.
Also, I think that there’s an even lower category than sentient. The example I like to use for this is a thermostat. It is agentic in that it is a system that responds behaviorally to changes in the environment (I’d call this a reflex perhaps, or stimulus/response pair), but it is not sentient because unlike a worm it doesn’t have a computational system that attaches valence to these reflexes. I think that there are entities which I would classify as living beings that fall into the non-sentient category. For example: I think probably coral polyps and maybe jellyfish have computational systems too simplistic for valence and thus respond purely reflexively. If this is the case, then I would not torture a single worm to save any number of coral polyps. I think most (non ML) computer programs fall into this category. I think a reinforcement learning agent transcends this category, by having valenced reactions to stimuli, and thus should be considered at least comparable to sentient beings like insects.
I like the distinctions you make between sentient, sapient, and conscious. I would like to bring up some thoughts about how to choose a morality that I think are relevant to your points about death of cows and transient beings, which I disagree with.
I think that when choosing our morality, we should do so under the assumption that we have been given complete omnipotent control over reality and that we should analyze all of our values independently, not taking into consideration any trade-offs, even when some of our values are logically impossible to satisfy simultaneously. Only after doing this do we start talking about what’s actually physically and logically possible and what trade-offs we are willing to make, while always making sure to be clear when something is actually part of our morality vs when something is a trade-off.
The reason for this approach is to avoid accidentally locking in trade-offs into our morality which might later turn out to not actually be necessary. And the great thing about it is that if we have not accidentally locked in any trade-offs into our morality, this approach should give back the exact same morality that we started off with, so when it doesn’t return the same answer I find it pretty instructive.
I think this applies to the idea that it’s okay to kill cows, because when I consider a world where I have to decide whether or not cows die, and this decision will not affect anything else in any way, then my intuition is that I slightly prefer that they not die. Therefore my morality is that cows should not die, even though in practice I think I might make similar trade-offs as you when it comes to cows in the world of today.
Something similar applies to transient computational subprocesses. If you had unlimited power and you had to explicitly choose if the things you currently call “transient computational subprocesses” are terminated, and you were certain that this choice would not affect anything else in any way at all (not even the things you think it’s logically impossible for it not to affect), would you still choose to terminate them? Remember that no matter what you choose here, you can still choose to trade things off the same way afterwards, so your answer doesn’t have to change your behavior in any way.
It’s possible that you still give the exact same answers with this approach, but I figure there’s a chance this might be helpful.
That’s an interesting way of reframing the issue. I’m honestly just not sure about all of this reasoning, and remain so after trying to think about it with your reframing, but I feel like this does shift my thinking a bit. Thanks.
I think probably it makes sense to try reasoning both with and without tradeoffs, and then comparing the results.