Yes, these values are all different from each other, but a crux is I don’t think that the differing values amongst humans are so distinct from paperclips that it’s worth it to blur the differences, especially with very strong optimization, though I agree that human values form a sort as in a set of objects, trivially.
I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
To my best (very uncertain) quess, human values are defined through some relation of states of consciousness to social dynamic?
I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
I mostly agree with this, with caveats that a paper-clip outcome can happen, but it isn’t very likely.
(For example, radical eco-green views where humans have to be extinct so nature can heal definitely exist, and would be a paper-clip outcome from my perspective).
I was also talking about very bad from the perspective of another human, since I think this is surprisingly important when dealing with AI safety.
Yes, these values are all different from each other, but a crux is I don’t think that the differing values amongst humans are so distinct from paperclips that it’s worth it to blur the differences, especially with very strong optimization, though I agree that human values form a sort as in a set of objects, trivially.
I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
To my best (very uncertain) quess, human values are defined through some relation of states of consciousness to social dynamic?
I mostly agree with this, with caveats that a paper-clip outcome can happen, but it isn’t very likely.
(For example, radical eco-green views where humans have to be extinct so nature can heal definitely exist, and would be a paper-clip outcome from my perspective).
I was also talking about very bad from the perspective of another human, since I think this is surprisingly important when dealing with AI safety.