Eliezer: “And you might not notice if your goals shifted only a bit at a time, as your emotional balance altered with the strange new harmonies of your brain.”
This is yet another example of Eliezer’s disagreement with the human race about morality. This actually happens to us all the time, without any modification at all, and we don’t care at all, and in fact we tend to be happy about it, because according to the new goal system, our goals have improved. So this suggests that we still won’t care if it happens due to upgrading.
Just so. But not many of us become full-blown psychotic sadists, and few indeed of those have godlike superpowers.
So I think that the not-particularly harmfulness of the usual range of moral self-modification is not a strong argument for letting rip with the self-enhancing drugs.
Learning new values as people naturally do is a very different thing than, say, deleting the empathy part of your brain and becoming a psychopath. The first are changes that we accept voluntarily for the most part, whereas the second no one would chose for themselves and you would be horrified at your future self if you did so.
The point is just because future you doesn’t care, doesn’t mean it’s a bad thing. An extreme example, if you were to just delete your intelligence entirely, you wouldn’t regret it. But I don’t think you want that for yourself.
There are less obvious cases, like deleting your value for social interaction and withdrawing from society completely. It’s not an obviously bad thing, but I don’t think it’s something you would choose voluntarily.
Eliezer: “And you might not notice if your goals shifted only a bit at a time, as your emotional balance altered with the strange new harmonies of your brain.”
This is yet another example of Eliezer’s disagreement with the human race about morality. This actually happens to us all the time, without any modification at all, and we don’t care at all, and in fact we tend to be happy about it, because according to the new goal system, our goals have improved. So this suggests that we still won’t care if it happens due to upgrading.
Just so. But not many of us become full-blown psychotic sadists, and few indeed of those have godlike superpowers.
So I think that the not-particularly harmfulness of the usual range of moral self-modification is not a strong argument for letting rip with the self-enhancing drugs.
Learning new values as people naturally do is a very different thing than, say, deleting the empathy part of your brain and becoming a psychopath. The first are changes that we accept voluntarily for the most part, whereas the second no one would chose for themselves and you would be horrified at your future self if you did so.
The point is just because future you doesn’t care, doesn’t mean it’s a bad thing. An extreme example, if you were to just delete your intelligence entirely, you wouldn’t regret it. But I don’t think you want that for yourself.
There are less obvious cases, like deleting your value for social interaction and withdrawing from society completely. It’s not an obviously bad thing, but I don’t think it’s something you would choose voluntarily.