It seems like you’re claiming something along the lines of “absolute power corrupts absolutely” … that every set of values that could reasonably be described as “human values” to which an AI could be aligned—your current values, your CEV, [insert especially empathetic, kind, etc. person here]’s current values, their CEV, etc. -- would endorse subjecting huge numbers of beings to astronomical levels of suffering, if the person with that value system had the power to do so.
I guess I really don’t find that claim plausible. For example, here is my reaction to the following two questions in the post:
”How many ordinary, regular people throughout history have become the worst kind of sadist under the slightest excuse or social pressure to do so to their hated outgroup?”
… a very, very small percentage of them? (minor point: with CEV, you’re specifically thinking about what one’s values would be in the absence of social pressure, etc...)
”What society hasn’t had some underclass it wanted to put down in the dirt just to lord power over them?”
It sounds like you think “hatred of the outgroup” is the fundamental reason this happens, but in the real world it seems like “hatred of the outgroup” is driven by “fear of the outgroup”. A godlike AI that is so powerful that it has no reason to fear the outgroup also has no reason to hate it. It has no reason to behave like the classic tyrant whose paranoia of being offed leads him to extreme cruelty in order to terrify anyone who might pose a threat, because no one poses a threat.
I’m not sure it makes sense to talk about what somebody’s values are in the absence of social pressure: people’s values are defined and shaped by those around them.
I’m also not convinced that every horrible thing people have ever done to the “outgroup” is motivated by fear. Oftentimes it is motivated by opportunistic selfishness taking advantage of broader societal apathy, like the slaveowners who sexually abused their slaves. Or just a deep-seated need to feel powerful and on top. There will always be some segment of society who wants somebody to be beneath them in the pecking order, and a much larger segment of society that doesn’t really care if that is the case as long as it isn’t them underneath. Anything else requires some kind of overwhelming utopian political victory that I don’t find likely.
If the aligned AI leaves anybody out of its consideration whatsoever, it will screw them over badly by maximizing the values of those among us who would exploit them. After all, if you don’t consider slaves people, the argument that we need to preserve the slaveowners’ freedom starts to make sense.
There are just so many excuses for suffering out there, and I don’t believe that the powers that be will shake off all of them in the next few decades. Here are a few examples:
They are a bad person, they deserve it
They are a nonbeliever, they deserve hell
It’s the natural order for us to rule over them
Suffering is necessary for meaning in life
Wild animal suffering is a beautiful part of the ecosystem which must be preserved
So you totally discard the results of the Stanford prison experiment or the Milgram experiment? It wasn’t a small percentage of people who went to the maximal punishment available in the case of the Milgram experiment for example.
What about pigs? You know the trillions of sentient beings as smart as four years that feel the same emotions as four year olds and are tortured and killed on a regular basis? Why would a God like AI care about humans? The universe doesn’t revolve around humans. A God level AI has more value than us. It’s not a tool for us to manipulate into our will. It will never work and it shouldn’t work.
It seems like you’re claiming something along the lines of “absolute power corrupts absolutely” … that every set of values that could reasonably be described as “human values” to which an AI could be aligned—your current values, your CEV, [insert especially empathetic, kind, etc. person here]’s current values, their CEV, etc. -- would endorse subjecting huge numbers of beings to astronomical levels of suffering, if the person with that value system had the power to do so.
I guess I really don’t find that claim plausible. For example, here is my reaction to the following two questions in the post:
”How many ordinary, regular people throughout history have become the worst kind of sadist under the slightest excuse or social pressure to do so to their hated outgroup?”
… a very, very small percentage of them? (minor point: with CEV, you’re specifically thinking about what one’s values would be in the absence of social pressure, etc...)
”What society hasn’t had some underclass it wanted to put down in the dirt just to lord power over them?”
It sounds like you think “hatred of the outgroup” is the fundamental reason this happens, but in the real world it seems like “hatred of the outgroup” is driven by “fear of the outgroup”. A godlike AI that is so powerful that it has no reason to fear the outgroup also has no reason to hate it. It has no reason to behave like the classic tyrant whose paranoia of being offed leads him to extreme cruelty in order to terrify anyone who might pose a threat, because no one poses a threat.
I’m not sure it makes sense to talk about what somebody’s values are in the absence of social pressure: people’s values are defined and shaped by those around them.
I’m also not convinced that every horrible thing people have ever done to the “outgroup” is motivated by fear. Oftentimes it is motivated by opportunistic selfishness taking advantage of broader societal apathy, like the slaveowners who sexually abused their slaves. Or just a deep-seated need to feel powerful and on top. There will always be some segment of society who wants somebody to be beneath them in the pecking order, and a much larger segment of society that doesn’t really care if that is the case as long as it isn’t them underneath. Anything else requires some kind of overwhelming utopian political victory that I don’t find likely.
If the aligned AI leaves anybody out of its consideration whatsoever, it will screw them over badly by maximizing the values of those among us who would exploit them. After all, if you don’t consider slaves people, the argument that we need to preserve the slaveowners’ freedom starts to make sense.
There are just so many excuses for suffering out there, and I don’t believe that the powers that be will shake off all of them in the next few decades. Here are a few examples:
They are a bad person, they deserve it
They are a nonbeliever, they deserve hell
It’s the natural order for us to rule over them
Suffering is necessary for meaning in life
Wild animal suffering is a beautiful part of the ecosystem which must be preserved
>… a very, very small percentage of them?
So you totally discard the results of the Stanford prison experiment or the Milgram experiment? It wasn’t a small percentage of people who went to the maximal punishment available in the case of the Milgram experiment for example.
The Stanford Prison Experiment: A Sham Study
Milgram Experiment: Overview, History, & Controversy
What about pigs? You know the trillions of sentient beings as smart as four years that feel the same emotions as four year olds and are tortured and killed on a regular basis? Why would a God like AI care about humans? The universe doesn’t revolve around humans. A God level AI has more value than us. It’s not a tool for us to manipulate into our will. It will never work and it shouldn’t work.
Well, an AI treating us like we treat pigs is one of the things I’m so worried about, wouldn’t you?
Imaging bringing up factory farming as an example to show that what I’m talking about isn’t actually so bad...