Freedom and utopia for all humans sounds great until the technology to create tailor-made sentient nonhumans comes along. Or hell, just the David Attenborough like desire to spectate the horrors of the nonhuman biosphere on Earth and billions of planets beyond. People’s values have proven horrible enough times to make me far more afraid of Utopia than any paperclip maximizer.
That’s why we need freedom and utopia for all living beings. Not just for all humans. Anthropocentrism is absurd and insane, much like the natural state with its endless death and suffering. Both must be abolished.
This post by the same author answers your comment: https://carado.moe/surprise-you-want.html Freedom is just a heuristic; let’s call the actual thing we want for humans our values (which is what we hope Elua will return in this scenario). By definition, our values are everything we want, including possibly the abolition of anthropocentrism. What is meant here by freedom and utopia is “the best scenario”. It’s not about what our values are, it’s about a method proposed to reach them.
Why would it harm humans? Do you think that the expected value of thinking about it is negative because of how it might lead us to overlook some forms of alignment?
Any insufficiently human-supremacist AI is an S-risk for humanity. Non-human entities are only valued inasmuch as individual humans value them concretely. No abstract preferences over them should be permitted.
See this sort of thing is why Clippy sounds relatively good to me, and why I don’t agree with Eliezer when he says humans all want the same thing and so CEV would be coherent when applied over all of humanity.
See this sort of thing is why Clippy sounds relatively good to me, and why I don’t agree with Eliezer when he says humans all want the same thing and so CEV would be coherent when applied over all of humanity.
This is a bit difficult to believe, has Eliezer really said something that absurd on-the-record and left it unretracted? Do you have a link?
When a paperclip maximizer and a pencil maximizer do different things, they are not disagreeing about anything, they are just different optimization processes. You cannot detach should-ness from any specific criterion of should-ness and be left with a pure empty should-ness that the paperclip maximizer and pencil maximizer can be said to disagree about—unless you cover “disagreement” to include differences where two agents have nothing to say to each other.
But this would be an extreme position to take with respect to your fellow humans, and I recommend against doing so. Even a psychopath would still be in a common moral reference frame with you, if, fully informed, they would decide to take a pill that would make them non-psychopaths. If you told me that my ability to care about other people was neurologically damaged, and you offered me a pill to fix it, I would take it. Now, perhaps some psychopaths would not be persuadable in-principle to take the pill that would, by our standards, “fix” them. But I note the possibility to emphasize what an extreme statement it is to say of someone:
“We have nothing to argue about, we are only different optimization processes.”
That should be reserved for paperclip maximizers, not used against humans whose arguments you don’t like.
-Yudkowsky 2008, Moral Error and Moral Disagreement
Seems to me to imply that everybody has basically the same values, that it is rare for humans to have irreconcilable moral differences. Also seems to me to be unfortunately and horribly wrong.
As for retraction I don’t know if he has changed his view on this, I only know it’s part of the Metaethics sequence.
The proposition is not that “everybody has basically the same values”, it’s more that everybody has basically the same brains, so a meeting of minds should ideally always be possible between humans, even if it doesn’t happen in practice.
And yet, as was pointed out in a Slate Star Codex thread once, nearly everyone has experiences that other people do not, including having access to entire distinct classes of qualia. The usual examples are that some people have and others lack internal dialogue, or the ability to visually imagine things.
In my case, I lack some of the social instincts neurotypicals take for granted, but on the other hand, I know exactly what divine possession feels like and what all the great mystics of history were babbling about, and most people don’t. And our brains aren’t similar enough for me to have much hope of getting people who cannot have that experience to value it.
The proposition is not that “everybody has basically the same values”, it’s more that everybody has basically the same brains, so a meeting of minds should ideally always be possible between humans, even if it doesn’t happen in practice.
No? There exist real living breathing humans that have radically altered brain structure, such as those with one hemisphere removed via surgical procedures or who have a dramatic brain injury.
It’s also not too difficult to imagine in the future with the possibilities of more advanced genetic engineering, there could be viable humans born with brains more similar to chimpanzees or dolphins than 2023 humans.
I could have sworn he said something in the sequences along the lines of “One might be tempted to say of our fellow humans, when arguing over morality, that they simply mean different things by morality and there is nothing factual to argue about, only an inevitable fight. This may be true of things like paperclip maximizers and alien minds. But it is not something that is true of our fellow humans.”
Unfortunately I cannot find it right now as I don’t remember the exact phrasing, but it stuck with me when I read it as obviously wrong. If anybody knows what quote I’m talking about please chime in.
This is one of the things I despise about this community. People here pretend to be altruists, but are not. It is incoherent to value humans and not to value the other beings we share the planet with who, in the space of minds, are massively closer to humans than they are to any AI we are likely to create. But you retreat to moral irrealism and the primacy of arbitrary whims (utility functions) above all else when faced with the supreme absurdity of human supremacy.
Freedom and utopia for all humans sounds great until the technology to create tailor-made sentient nonhumans comes along. Or hell, just the David Attenborough like desire to spectate the horrors of the nonhuman biosphere on Earth and billions of planets beyond. People’s values have proven horrible enough times to make me far more afraid of Utopia than any paperclip maximizer.
That’s why we need freedom and utopia for all living beings. Not just for all humans. Anthropocentrism is absurd and insane, much like the natural state with its endless death and suffering. Both must be abolished.
This post by the same author answers your comment: https://carado.moe/surprise-you-want.html
Freedom is just a heuristic; let’s call the actual thing we want for humans our values (which is what we hope Elua will return in this scenario). By definition, our values are everything we want, including possibly the abolition of anthropocentrism.
What is meant here by freedom and utopia is “the best scenario”. It’s not about what our values are, it’s about a method proposed to reach them.
I’ve read that post before. I dislike its narcissistic implications. Even if true, it’s something I think humans can only be harmed by thinking about.
Why would it harm humans?
Do you think that the expected value of thinking about it is negative because of how it might lead us to overlook some forms of alignment?
Any insufficiently human-supremacist AI is an S-risk for humanity. Non-human entities are only valued inasmuch as individual humans value them concretely. No abstract preferences over them should be permitted.
See this sort of thing is why Clippy sounds relatively good to me, and why I don’t agree with Eliezer when he says humans all want the same thing and so CEV would be coherent when applied over all of humanity.
This is a bit difficult to believe, has Eliezer really said something that absurd on-the-record and left it unretracted? Do you have a link?
https://www.lesswrong.com/posts/BkkwXtaTf5LvbA6HB/moral-error-and-moral-disagreement
-Yudkowsky 2008, Moral Error and Moral Disagreement
Seems to me to imply that everybody has basically the same values, that it is rare for humans to have irreconcilable moral differences. Also seems to me to be unfortunately and horribly wrong.
As for retraction I don’t know if he has changed his view on this, I only know it’s part of the Metaethics sequence.
Wow, this does sound like unhinged nonsense. If he still maintains it circa 2023 then I would be really surprised.
The proposition is not that “everybody has basically the same values”, it’s more that everybody has basically the same brains, so a meeting of minds should ideally always be possible between humans, even if it doesn’t happen in practice.
And yet, as was pointed out in a Slate Star Codex thread once, nearly everyone has experiences that other people do not, including having access to entire distinct classes of qualia. The usual examples are that some people have and others lack internal dialogue, or the ability to visually imagine things.
In my case, I lack some of the social instincts neurotypicals take for granted, but on the other hand, I know exactly what divine possession feels like and what all the great mystics of history were babbling about, and most people don’t. And our brains aren’t similar enough for me to have much hope of getting people who cannot have that experience to value it.
No? There exist real living breathing humans that have radically altered brain structure, such as those with one hemisphere removed via surgical procedures or who have a dramatic brain injury.
For example, there’s the quite well known Phineas Gage: https://en.wikipedia.org/wiki/Phineas_Gage
It’s also not too difficult to imagine in the future with the possibilities of more advanced genetic engineering, there could be viable humans born with brains more similar to chimpanzees or dolphins than 2023 humans.
I could haveswornhe said something in the sequences along the lines of “One might be tempted to say of our fellow humans, when arguing over morality, that they simply mean different things by morality and there is nothing factual to argue about, only an inevitable fight. This may be true of things like paperclip maximizers and alien minds. But it is not something that is true of our fellow humans.”Unfortunately I cannot find it right now as I don’t remember the exact phrasing, but it stuck with me when I read it as obviously wrong. If anybody knows what quote I’m talking about please chime in.Edit: Found it, see other reply
Links to the original 2004 article on intelligence.org seem to be broken...they are not even 404ing.
This is one of the things I despise about this community. People here pretend to be altruists, but are not. It is incoherent to value humans and not to value the other beings we share the planet with who, in the space of minds, are massively closer to humans than they are to any AI we are likely to create. But you retreat to moral irrealism and the primacy of arbitrary whims (utility functions) above all else when faced with the supreme absurdity of human supremacy.