Eli: “I do not assert that all humans end up in the same moral frame of reference (with regard to any particular extrapolation method). I do think that psychological unity is typically underestimated,”
- right, thanks for the clarification.
“But if you read “Coherent Extrapolated Volition” you’ll see that it’s specifically designed to handle, among other problems, the problem of, “What if we don’t all want the same thing?” What then can an AI programmer do that does not constitute being a jerk? That was my attempt to answer.”
- this is where I think I disagree with you somewhat. Your notion of what constitutes a “jerk” thing to do differs from my notion of what constitutes a jerk thing to do. I would not extrapolate the volitions of people whose volitions I deem to be particularly dangerous, in fact I would probably only extrapolate the volition of a small subset (perhaps 1 thousand − 1 million) people whose outward philosophical stances on life were at least fairly similar to mine. I would consider it a jerk-like thing to extrapolate, with equal weight, the volitions of every human on the planet, including all the religious fundamentalists, etc. In any case, the problem for you is that if you and I disagree on exactly whose volitions should be taken into account in the CEV algorithm, how exactly do we settle our dispute? You can’t appeal to the psychological unity of humankind, for I am human and I disagree with you. You’re an antirealist, so there’s no objective fact of the matter either.
Or are you a closet realist? Is there some objective notion of “non-jerkness” that transcends the views of any particular human?
As a realist, I don’t have this problem. I think I have found objective criteria upon which to judge the question.
Eli: “I do not assert that all humans end up in the same moral frame of reference (with regard to any particular extrapolation method). I do think that psychological unity is typically underestimated,”
- right, thanks for the clarification.
“But if you read “Coherent Extrapolated Volition” you’ll see that it’s specifically designed to handle, among other problems, the problem of, “What if we don’t all want the same thing?” What then can an AI programmer do that does not constitute being a jerk? That was my attempt to answer.”
- this is where I think I disagree with you somewhat. Your notion of what constitutes a “jerk” thing to do differs from my notion of what constitutes a jerk thing to do. I would not extrapolate the volitions of people whose volitions I deem to be particularly dangerous, in fact I would probably only extrapolate the volition of a small subset (perhaps 1 thousand − 1 million) people whose outward philosophical stances on life were at least fairly similar to mine. I would consider it a jerk-like thing to extrapolate, with equal weight, the volitions of every human on the planet, including all the religious fundamentalists, etc. In any case, the problem for you is that if you and I disagree on exactly whose volitions should be taken into account in the CEV algorithm, how exactly do we settle our dispute? You can’t appeal to the psychological unity of humankind, for I am human and I disagree with you. You’re an antirealist, so there’s no objective fact of the matter either.
Or are you a closet realist? Is there some objective notion of “non-jerkness” that transcends the views of any particular human?
As a realist, I don’t have this problem. I think I have found objective criteria upon which to judge the question.