Steven: quite possibly related. I don’t think they’re exactly the same (the classic comic book/high fantasy “I’m evil and I know it” villain fits A2, but I’d describe him as amoral), but it’s an interesting parallel.
Eliezer: I’m coming more and more to the conclusion that our main area of disagreement is our willingness to believe that someone who disagrees with us really “embodies a different optimization process.” There are infinitely many self-consistent belief systems and infinitely many internally consistent optimization processes; while I believe mine to be the best I’ve found, I remain aware that if I held any of the others I would believe exactly the same thing. And that I would have no way of convincing the anti-Occam intelligence that Occam’s Razor was a good heuristic, or of convincing the psychopath who really doesn’t care about other people that he ‘ought’ to. So I hesitate to say that I’m right in any objective sense, since I’m not sure exactly what standard I’m pointing to when I say ‘objective.’
And I’ve had extended moral conversations with a few different people that led to us, eventually, concluding that our premises were so radically different that we really couldn’t have a sensible moral conversation. (to wit: I think my highest goal in life is to make myself happy. Because I’m not a sociopath making myself happy tends to involve having friends and making them happy. But the ultimate goal is me. Makes it hard to talk to someone who actually believes in some form of altruism).
Steven: quite possibly related. I don’t think they’re exactly the same (the classic comic book/high fantasy “I’m evil and I know it” villain fits A2, but I’d describe him as amoral), but it’s an interesting parallel.
Eliezer: I’m coming more and more to the conclusion that our main area of disagreement is our willingness to believe that someone who disagrees with us really “embodies a different optimization process.” There are infinitely many self-consistent belief systems and infinitely many internally consistent optimization processes; while I believe mine to be the best I’ve found, I remain aware that if I held any of the others I would believe exactly the same thing. And that I would have no way of convincing the anti-Occam intelligence that Occam’s Razor was a good heuristic, or of convincing the psychopath who really doesn’t care about other people that he ‘ought’ to. So I hesitate to say that I’m right in any objective sense, since I’m not sure exactly what standard I’m pointing to when I say ‘objective.’
And I’ve had extended moral conversations with a few different people that led to us, eventually, concluding that our premises were so radically different that we really couldn’t have a sensible moral conversation. (to wit: I think my highest goal in life is to make myself happy. Because I’m not a sociopath making myself happy tends to involve having friends and making them happy. But the ultimate goal is me. Makes it hard to talk to someone who actually believes in some form of altruism).