I didn’t read most of this post but i would like to point out one sentence from it.
Nearly all of us net prefer to be alive rather than dead, and so we can infer that these years of existence, experienced by humans, are net happy years.
Be careful with this reasoning. If you think it is reasonable to infer whether human lives are happy by humans’ self-judgement alone, then you can also use the same line of reasoning on AI. Hence if you can design an AI to be happy of their circumstance (as Anthropic’s model welfare efforts are already trying to do) then they’d also be living happy lives even if they’re sentient, which in turn invalidates the main argument of this post.
I feel the “make them happy with it” issue is covered implicitly by What is Evil about creating House Elves? where I am still reasonably happy with my answer, which, in a nutshell, is about the value of keeping personhood “tidy” as a concept… so that it is not worn down by the vicissitudes of of politics, economics, and culture in the potentially arbitrarily weird transhuman future.
16 years ago I offered a little science-fiction-story-premise like so:
Now imagine that humans create house elves to be capable of speech and eye contact and geometric verbal reasoning and laughter and so on. Only then does this two-species composite meet “radically different aliens”.
From the alien’s perspective, humans and house elves are nearly identical except for a small fudge, right? Since the humans were OK creating the house elves they must endorse that state as “theoretically acceptable”. Therefore it wouldn’t seem like that large of an imposition to ask the humans to modify themselves that way, right? Maybe the house elves are actually happier? And they’re certainly cheaper to feed!
In the same vein, however… going beyond the “disgustingness” of creating House Elves on purpose...
...there is a deeper issue here, which is that it might have happened naturally!
This idea used to be articulated quite well, and very thoroughly, over the course of MANY essays by Sister Y on the blog The View From Hell, that evolution itself plausibly hacked our minds to be happy with living even if our unhacked minds would rationally look at human life and say “no thank you”.
We might have a non-trivial number of “counterfactual ancestors” whose genomes gave them much more general and coherent faculties of reason, but then these faculties might have noticed that life itself is hellish, and caused them to agentically and rationally kill themselves on purpose for reasons they endorsed.
If this happened, then we should eventually be able to look at human genomes and detect the signature of this selection pressure, by reading the semantics of the genes and their effects on the brain, and checking for selection signatures in the patterns of synonymous mutations… and then we might face a deep question about what to do after that.
My naive opinion, without thinking about it SUPER a lot, is that I would like to have the kind of genome that can see clearly, and if this leads me to deem life too terrible to bear…
...and then I would try really hard to improve my life in objectively good waysbefore I opted out?
And, indeed, the only essays left up (and not self censored) on The View From Hell are six essays about the methods that Sister Y seems (if we read between the lines) to have deemed useful as “palliative advice on tolerating the inherent/structural sadness of life”: books, exercise, THC, demandingness, and two posts on how to have a good marriage despite modern civilization making this hard.
My real actual response would probably depend a lot on exactly how our thoughts turn out to be systematically censored, if they are even censored?
My guess is that most of the selection effects here will have occurred around the advent of agriculture (and its massive increase in human slavery, with many humans probably choosing to suicide rather than endure enslavement) and perhaps also a bunch around the time that humans gained symbolic reasoning powers and started burying their dead with ceremony, which implies an awareness of death, and the capacity to grieve, and a yearning for something-like-Heaven, which also implies awareness that “this world is not Heaven” (yet! (grown mindset (except back then they had no rational basis for thinking Heaven might be physically possible to realize, because transhumanism and nuclear power and automation and law & order and so on hadn’t been invented yet))).
I didn’t read most of this post but i would like to point out one sentence from it.
Be careful with this reasoning. If you think it is reasonable to infer whether human lives are happy by humans’ self-judgement alone, then you can also use the same line of reasoning on AI. Hence if you can design an AI to be happy of their circumstance (as Anthropic’s model welfare efforts are already trying to do) then they’d also be living happy lives even if they’re sentient, which in turn invalidates the main argument of this post.
I appreciate your attention to this detail!
I feel the “make them happy with it” issue is covered implicitly by What is Evil about creating House Elves? where I am still reasonably happy with my answer, which, in a nutshell, is about the value of keeping personhood “tidy” as a concept… so that it is not worn down by the vicissitudes of of politics, economics, and culture in the potentially arbitrarily weird transhuman future.
16 years ago I offered a little science-fiction-story-premise like so:
In the same vein, however… going beyond the “disgustingness” of creating House Elves on purpose...
...there is a deeper issue here, which is that it might have happened naturally!
This idea used to be articulated quite well, and very thoroughly, over the course of MANY essays by Sister Y on the blog The View From Hell, that evolution itself plausibly hacked our minds to be happy with living even if our unhacked minds would rationally look at human life and say “no thank you”.
We might have a non-trivial number of “counterfactual ancestors” whose genomes gave them much more general and coherent faculties of reason, but then these faculties might have noticed that life itself is hellish, and caused them to agentically and rationally kill themselves on purpose for reasons they endorsed.
If this happened, then we should eventually be able to look at human genomes and detect the signature of this selection pressure, by reading the semantics of the genes and their effects on the brain, and checking for selection signatures in the patterns of synonymous mutations… and then we might face a deep question about what to do after that.
My naive opinion, without thinking about it SUPER a lot, is that I would like to have the kind of genome that can see clearly, and if this leads me to deem life too terrible to bear…
...and then I would try really hard to improve my life in objectively good ways before I opted out?
And, indeed, the only essays left up (and not self censored) on The View From Hell are six essays about the methods that Sister Y seems (if we read between the lines) to have deemed useful as “palliative advice on tolerating the inherent/structural sadness of life”: books, exercise, THC, demandingness, and two posts on how to have a good marriage despite modern civilization making this hard.
My real actual response would probably depend a lot on exactly how our thoughts turn out to be systematically censored, if they are even censored?
My guess is that most of the selection effects here will have occurred around the advent of agriculture (and its massive increase in human slavery, with many humans probably choosing to suicide rather than endure enslavement) and perhaps also a bunch around the time that humans gained symbolic reasoning powers and started burying their dead with ceremony, which implies an awareness of death, and the capacity to grieve, and a yearning for something-like-Heaven, which also implies awareness that “this world is not Heaven” (yet! (grown mindset (except back then they had no rational basis for thinking Heaven might be physically possible to realize, because transhumanism and nuclear power and automation and law & order and so on hadn’t been invented yet))).