I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
To my best (very uncertain) quess, human values are defined through some relation of states of consciousness to social dynamic?
I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
I mostly agree with this, with caveats that a paper-clip outcome can happen, but it isn’t very likely.
(For example, radical eco-green views where humans have to be extinct so nature can heal definitely exist, and would be a paper-clip outcome from my perspective).
I was also talking about very bad from the perspective of another human, since I think this is surprisingly important when dealing with AI safety.
I think the easy difference is that totally optimized according to someone’s values world is going to be either very good (even if not perfect) or very bad from perspective of another human? I wouldn’t say it’s impossible, but it should be very specific combination of human values to make it just as valuable as turning everything into paperclips, not worse, not better.
To my best (very uncertain) quess, human values are defined through some relation of states of consciousness to social dynamic?
I mostly agree with this, with caveats that a paper-clip outcome can happen, but it isn’t very likely.
(For example, radical eco-green views where humans have to be extinct so nature can heal definitely exist, and would be a paper-clip outcome from my perspective).
I was also talking about very bad from the perspective of another human, since I think this is surprisingly important when dealing with AI safety.