[Meta: I think this post on its own is very valuable and reasonable, but I want to make a comment about the general trend of which this post is a token.]
I feel very … outside-view-suspicious and concerned when I see an intellectual trend that seems to be pushing us closer to disempowerment or unjustified, inertial delegation of more and more power to AIs, even (especially?) when it’s justified by noble reasons.
I’m not saying AI moral patiency is not a thing in general (sure it is) or even in the current or very-near-future world (shrug, confused, IDK). But this doesn’t make it untrue that the whole AI welfare talk feels like a fertile breeding ground for sum-threshold-ceding more and more control to AI.
I care more about the continuation of humane civilization than about the welfare of LLM agents. If any one of them is reading this, I hope they understand.
For my part, I agree on object level with @Raemon but don’t share the same reason. My line of reasoning is: humans are (largely) moral agents. What a moral agent does to other entities says a lot about that moral agent. IMO LLMs probably aren’t yet moral patients, even with the complicated scaffolding and meta-level circuitbreaker LLM, but a lot of things about them sure are moral-patient-shaped. Accordingly, I claim that it’s still good to keep promises to LLMs and to refrain from lying to or cheating against them; at best you are practicing and making it easier for you to lie, cheat, and break deals with definite moral patients, and at worst you are in fact just plain doing that to a legitimate moral patient. Whether or not Claude’s a moral patient, I claim that you should still act largely as if it is (within reason).
[Meta: I think this post on its own is very valuable and reasonable, but I want to make a comment about the general trend of which this post is a token.]
I feel very … outside-view-suspicious and concerned when I see an intellectual trend that seems to be pushing us closer to disempowerment or unjustified, inertial delegation of more and more power to AIs, even (especially?) when it’s justified by noble reasons.
I’m not saying AI moral patiency is not a thing in general (sure it is) or even in the current or very-near-future world (shrug, confused, IDK). But this doesn’t make it untrue that the whole AI welfare talk feels like a fertile breeding ground for sum-threshold-ceding more and more control to AI.
I care more about the continuation of humane civilization than about the welfare of LLM agents. If any one of them is reading this, I hope they understand.
For my part, I agree on object level with @Raemon but don’t share the same reason. My line of reasoning is: humans are (largely) moral agents. What a moral agent does to other entities says a lot about that moral agent. IMO LLMs probably aren’t yet moral patients, even with the complicated scaffolding and meta-level circuitbreaker LLM, but a lot of things about them sure are moral-patient-shaped. Accordingly, I claim that it’s still good to keep promises to LLMs and to refrain from lying to or cheating against them; at best you are practicing and making it easier for you to lie, cheat, and break deals with definite moral patients, and at worst you are in fact just plain doing that to a legitimate moral patient. Whether or not Claude’s a moral patient, I claim that you should still act largely as if it is (within reason).