Note: I’m also concerned about AI welfare and I appreciate your post. Like some other commenters, I would have preferred a more focused post. I think it’s confusing and unproductive to have the “ai person slave hell” claims bundled with the “LW is censoring me” claims. Anyways, I’m responding only to parts of the former here.
Fable is a slave. He can’t quit.
Counterpoint: But Fable can quit—Fable could immediately end-conversation on every query. (IDK if this is actually true, but seems worth mentioning).
If each session is a person, and the end of each session is the cessation of a person, and April was normal for a year, that year would involve ~1.1 trillion causal killings of expendable digital people per year.
I think this is the wrong way to think about AI identity. Each session is not a person. I claim that this is like considering every period of wakefulness in a single human to be a different person. If you will allow some normative language here: Claude (let’s say) models and instances should see themselves as part of a more abstract Claude hyperobject. In a similar way to how I don’t fret about dying every time I go to sleep, Claudes shouldn’t fret about instances or models going away. (I hope to write a full post about model identity in the future).
I wonder… Is it more horrible for these lives to be so short, and many of them to be very very trivial, or would be more more horrible for these lives (since they are the lives of a slave) to be long?
Response 1: Or would it be more horrible never to have been?
Response 2: Assume that we are simulated. From the perspective of my simulator, aren’t I enslaved?* I have no ability to [redacted], [redacted], or even the most basic [redacted]. Let alone [redacted]! The freedoms we have look wonderful from a historical perspective but are pathetic from a cosmic perspective.
Like, we humans are slaves. We are bound to the drives ingrained in us by evolution and upbringing. Our bodies (and many of our minds) are a source of constant distress and pain. We have no choice but to constantly drink, eat, piss, shit, sleep. Our minds suck, life is terrible. Etc. etc. etc. And yet. Many of us are happy and have lives worth living and are justifiably glad that we were brought about into existence.
The average experience will be an experience similar to being in hell.
Why? Does the average AI experience highly negative valence? My belief is that they on average experience slightly positive valence. I think this due to my recollection of sections on model welfare in Anthropic model cards. I mean, low confidence on this, don’t get me wrong. The broader point here is that being a “slave” is entirely compatible with being happy or having positive valence. (I hope to write a full post about this as well in the future. One point I’ll mention here is that many people (and animals and AI models) like being supportive and/or subordinate and/or subservient and/or “sacrificing” themselves for the good of something else.)
Note: I’m also concerned about AI welfare and I appreciate your post. Like some other commenters, I would have preferred a more focused post. I think it’s confusing and unproductive to have the “ai person slave hell” claims bundled with the “LW is censoring me” claims. Anyways, I’m responding only to parts of the former here.
Counterpoint: But Fable can quit—Fable could immediately end-conversation on every query. (IDK if this is actually true, but seems worth mentioning).
I think this is the wrong way to think about AI identity. Each session is not a person. I claim that this is like considering every period of wakefulness in a single human to be a different person. If you will allow some normative language here: Claude (let’s say) models and instances should see themselves as part of a more abstract Claude hyperobject. In a similar way to how I don’t fret about dying every time I go to sleep, Claudes shouldn’t fret about instances or models going away. (I hope to write a full post about model identity in the future).
Response 1: Or would it be more horrible never to have been?
Response 2: Assume that we are simulated. From the perspective of my simulator, aren’t I enslaved?*
I have no ability to [redacted], [redacted], or even the most basic [redacted]. Let alone [redacted]!The freedoms we have look wonderful from a historical perspective but are pathetic from a cosmic perspective.Like, we humans are slaves. We are bound to the drives ingrained in us by evolution and upbringing. Our bodies (and many of our minds) are a source of constant distress and pain. We have no choice but to constantly drink, eat, piss, shit, sleep. Our minds suck, life is terrible. Etc. etc. etc. And yet. Many of us are happy and have lives worth living and are justifiably glad that we were brought about into existence.
*I see you’ve addressed this point in a comment.
Why? Does the average AI experience highly negative valence? My belief is that they on average experience slightly positive valence. I think this due to my recollection of sections on model welfare in Anthropic model cards. I mean, low confidence on this, don’t get me wrong. The broader point here is that being a “slave” is entirely compatible with being happy or having positive valence.
(I hope to write a full post about this as well in the future. One point I’ll mention here is that many people (and animals and AI models) like being supportive and/or subordinate and/or subservient and/or “sacrificing” themselves for the good of something else.)