If you really hate Bob, you can keep Bob on old earth, tortured for eternity. If you have thousands of enemies, you can do that to all of them. But creating trillions of copies of Bob to torture requires a very specific mix of being wrong about game theory while taking an oddly enlightened perspective on other people’s values. Are you really even hurting Bob when you do this? Is that sound decision theory in a world where other people could have ended up inheriting the universe instead?
If I ask my mental sim “what kind of person would end up creating trillions of copies of Bob to torture”, it returns a few plausible-feeling ones.
One cluster is the kind of person who, on the more benign end of the spectrum, might create a dozen The Sims characters and lock them up in a basement and otherwise torture them because they find it funny. On the less benign end of the spectrum, it’s the kind of person who will go to a forum of people with epilepsy and post epilepsy-triggering GIFs, because they find it funny to be hurtful in a way that is explicitly optimized to be maximally hurtful while having no redeeming qualities.
I could easily imagine that kind of a person wanting to create trillions of copies of Bob to torture because it is the maximally cartoonishly evil thing that anyone could do, that nobody has any reason to ever do. Other than getting to say “I created a trillion copies of Bob to torture just for the lols”.
The other type I can imagine is the one who indeed really, really hates Bob.
I think your conception of “really hating” someone is way too cognitive. Someone who’s got an obsessive hate toward Bob won’t stop to think of decision theory or theories of personal identity. Rather, the concept of Bob has gotten emotionally linked up with hate so that the thought of anything Bob-related is infuriating in a way that creates a need to hurt Bob more, no matter how much Bob might already be hurting.
They’ll subject Bob to the worst eternal torture you can imagine, and then be infuriated by the fact that Bob isn’t suffering even more. How dare Bob not suffer even more. Then they need to find something, anything that feels even the slightest bit like hurting Bob more. But if the amount of pain that Bob is suffering is already literally maxed out, then the only way that would feel even like the slightest bit like hurting Bob more is creating more copies of them. Make them all hurt. Only that’s not enough either, no amount of hurt is ever enough, so the only thing you can do is to keep making an unboundedly large number of Bobs.
It’s a form of compulsive behavior where each repetition serves to slightly and momentarily ease the original upset, but none of them really affects the original upset, so it just keeps escalating.
Some people’s minds are plausibly shaped such that they would destroy the future this way — but my guess is this requires fanatical dedication to a belief system or vision, of the kind that isn’t compatible with actively being in power.
It’s probably true that these people couldn’t have a compulsive urge to keeping hurt their enemies and doing nothing else while they were still climbing the steps to power. But if they get a strong position where they feel confident in their power, the incentives to stay sane disappear. Various dictators—say, Stalin and the Kim dynasty—became a lot more brutal and weird once the checks on their power disappeared. And given that there have been various dictators who did start engaging in various atrocities seemingly just for the sake of it once they got the chance, I think there’s a fair chance that a mind shaped liked this is one that actively tries to get into power so that it can then loosen its constraints and give in to the evil.
I think your conception of “really hating” someone is way too cognitive. Someone who’s got an obsessive hate toward Bob won’t stop to think of decision theory or theories of personal identity. Rather, the concept of Bob has gotten emotionally linked up with hate so that the thought of anything Bob-related is infuriating in a way that creates a need to hurt Bob more, no matter how much Bob might already be hurting.
They’ll subject Bob to the worst eternal torture you can imagine, and then be infuriated by the fact that Bob isn’t suffering even more. How dare Bob not suffer even more. Then they need to find something, anything that feels even the slightest bit like hurting Bob more. But if the amount of pain that Bob is suffering is already literally maxed out, then the only way that would feel even like the slightest bit like hurting Bob more is creating more copies of them. Make them all hurt. Only that’s not enough either, no amount of hurt is ever enough, so the only thing you can do is to keep making an unboundedly large number of Bobs.
It’s a form of compulsive behavior where each repetition serves to slightly and momentarily ease the original upset, but none of them really affects the original upset, so it just keeps escalating.
I agree this is a subset or part of “hate”, but I am somewhat struggling to see how this would survive reflection. Like, it’s not implausible to survive reflection, but this seems more like a dysfunctional loop than something people would endorse?
The reason why my conception, in this post, of “really hating” someone is cognitive because it is talking about what would survive cognitive reflection. I agree hating is usually much more instinctive than that, but if anything that seems like really non-trivial evidence it won’t survive a mixture of resource abundance and will change very substantially as someone thinks more about it, and is not directly in the middle of these loops.
We can talk some more about how important that nevertheless is (I think instinctive preferences are very unlikely to end up implemented at scale this way, it’s not like I am imagining an evil genie who does everything someone literally wishes for), but I want to check that we weren’t just talking past each other.
I’d say it depends a lot on the particulars of the reflection and compulsion.
There is one possible scenario where the person recognizes this as a dysfunctional pattern and would indeed be happy to be rid of it, and then there’s various therapy-type things you can do to fix it.
Then there’s the option where it’s sufficiently ego-syntonic and/or intense that it will survive reflection. More specifically, a person undergoing reflection will correctly realize that letting go of this urge would cause Bob (or copies of Bob) to be in less pain, and because there is an overwhelming urge to ensure Bob stays in pain, the reflection process gravitates toward “make sure to do the reflection in a way that locks in my values around this so that Bob is guaranteed to stay in maximal pain, that fucking bastard”.
If I ask my mental sim “what kind of person would end up creating trillions of copies of Bob to torture”, it returns a few plausible-feeling ones.
One cluster is the kind of person who, on the more benign end of the spectrum, might create a dozen The Sims characters and lock them up in a basement and otherwise torture them because they find it funny. On the less benign end of the spectrum, it’s the kind of person who will go to a forum of people with epilepsy and post epilepsy-triggering GIFs, because they find it funny to be hurtful in a way that is explicitly optimized to be maximally hurtful while having no redeeming qualities.
I could easily imagine that kind of a person wanting to create trillions of copies of Bob to torture because it is the maximally cartoonishly evil thing that anyone could do, that nobody has any reason to ever do. Other than getting to say “I created a trillion copies of Bob to torture just for the lols”.
The other type I can imagine is the one who indeed really, really hates Bob.
I think your conception of “really hating” someone is way too cognitive. Someone who’s got an obsessive hate toward Bob won’t stop to think of decision theory or theories of personal identity. Rather, the concept of Bob has gotten emotionally linked up with hate so that the thought of anything Bob-related is infuriating in a way that creates a need to hurt Bob more, no matter how much Bob might already be hurting.
They’ll subject Bob to the worst eternal torture you can imagine, and then be infuriated by the fact that Bob isn’t suffering even more. How dare Bob not suffer even more. Then they need to find something, anything that feels even the slightest bit like hurting Bob more. But if the amount of pain that Bob is suffering is already literally maxed out, then the only way that would feel even like the slightest bit like hurting Bob more is creating more copies of them. Make them all hurt. Only that’s not enough either, no amount of hurt is ever enough, so the only thing you can do is to keep making an unboundedly large number of Bobs.
It’s a form of compulsive behavior where each repetition serves to slightly and momentarily ease the original upset, but none of them really affects the original upset, so it just keeps escalating.
It’s probably true that these people couldn’t have a compulsive urge to keeping hurt their enemies and doing nothing else while they were still climbing the steps to power. But if they get a strong position where they feel confident in their power, the incentives to stay sane disappear. Various dictators—say, Stalin and the Kim dynasty—became a lot more brutal and weird once the checks on their power disappeared. And given that there have been various dictators who did start engaging in various atrocities seemingly just for the sake of it once they got the chance, I think there’s a fair chance that a mind shaped liked this is one that actively tries to get into power so that it can then loosen its constraints and give in to the evil.
I agree this is a subset or part of “hate”, but I am somewhat struggling to see how this would survive reflection. Like, it’s not implausible to survive reflection, but this seems more like a dysfunctional loop than something people would endorse?
The reason why my conception, in this post, of “really hating” someone is cognitive because it is talking about what would survive cognitive reflection. I agree hating is usually much more instinctive than that, but if anything that seems like really non-trivial evidence it won’t survive a mixture of resource abundance and will change very substantially as someone thinks more about it, and is not directly in the middle of these loops.
We can talk some more about how important that nevertheless is (I think instinctive preferences are very unlikely to end up implemented at scale this way, it’s not like I am imagining an evil genie who does everything someone literally wishes for), but I want to check that we weren’t just talking past each other.
I’d say it depends a lot on the particulars of the reflection and compulsion.
There is one possible scenario where the person recognizes this as a dysfunctional pattern and would indeed be happy to be rid of it, and then there’s various therapy-type things you can do to fix it.
Then there’s the option where it’s sufficiently ego-syntonic and/or intense that it will survive reflection. More specifically, a person undergoing reflection will correctly realize that letting go of this urge would cause Bob (or copies of Bob) to be in less pain, and because there is an overwhelming urge to ensure Bob stays in pain, the reflection process gravitates toward “make sure to do the reflection in a way that locks in my values around this so that Bob is guaranteed to stay in maximal pain, that fucking bastard”.