If the Nazis are unable to change their terminal values, then Good|Nazi has a substantial difference compared to what we mean when we say Good. Nazis might use the same word, or it might translate as “the same.” It might even be similar along many dimensions. Good|Jew might be the same as Good (they don’t seem substantially different then humans) although this isn’t required by the problem, but Good|Nazi ends up being something that I just don’t care about in the case where we are talking about exterminating Jews.
There might be other conditions where Good and Good|Nazi overlap, and in those cases I probably would agree that the Nazi should do the Good thing, which would also be to do the Good|Nazi thing. But I don’t have any reason to favor Good|Nazi over Good, and so where they differ (the extermination of Jews) I am unmotivated to defend or even to allow the Good|Nazi point of view to have a say in what is going on.
You indeed needn’t care about “good|Nazi”, but the important question in this hypothetical is whether you care about “happy|Nazi” or “suffer|Nazi”. I don’t care much whether the outcome is considered good by someone else, the less so if that person is evil, but still it could bother me if the outcome causes that person to suffer.
I don’t particularly want “suffer|Nazi” at least in and of itself.
But it works out the same way. A mosquito might suffer from not drinking my blood. That doesn’t mean I will just let it. A paperclip maximizer might be said to suffer from not getting to turn the planet into paperclips, if it were restrained.
If the only way to end suffer|Nazi is to violate what’s Good, then I am actually pretty okay with suffer|Nazi as an outcome. I’d still prefer ((happy|Nazi) & Good) to ((suffer|Nazi) & Good), but I see no problem with ((suffer|Nazi) & Good) winning out over ((happy|Nazi) & Bad). My preference for things with differing value systems not to suffer does not override my value system in and of itself.
If the Nazis are unable to change their terminal values, then Good|Nazi has a substantial difference compared to what we mean when we say Good. Nazis might use the same word, or it might translate as “the same.” It might even be similar along many dimensions. Good|Jew might be the same as Good (they don’t seem substantially different then humans) although this isn’t required by the problem, but Good|Nazi ends up being something that I just don’t care about in the case where we are talking about exterminating Jews.
There might be other conditions where Good and Good|Nazi overlap, and in those cases I probably would agree that the Nazi should do the Good thing, which would also be to do the Good|Nazi thing. But I don’t have any reason to favor Good|Nazi over Good, and so where they differ (the extermination of Jews) I am unmotivated to defend or even to allow the Good|Nazi point of view to have a say in what is going on.
You indeed needn’t care about “good|Nazi”, but the important question in this hypothetical is whether you care about “happy|Nazi” or “suffer|Nazi”. I don’t care much whether the outcome is considered good by someone else, the less so if that person is evil, but still it could bother me if the outcome causes that person to suffer.
I don’t particularly want “suffer|Nazi” at least in and of itself.
But it works out the same way. A mosquito might suffer from not drinking my blood. That doesn’t mean I will just let it. A paperclip maximizer might be said to suffer from not getting to turn the planet into paperclips, if it were restrained.
If the only way to end suffer|Nazi is to violate what’s Good, then I am actually pretty okay with suffer|Nazi as an outcome. I’d still prefer ((happy|Nazi) & Good) to ((suffer|Nazi) & Good), but I see no problem with ((suffer|Nazi) & Good) winning out over ((happy|Nazi) & Bad). My preference for things with differing value systems not to suffer does not override my value system in and of itself.