(1): I think it’s a prominent naturalistic feature; as EY said above, in a physical universe there are only quantum amplitudes, and if two agents have sufficiently accurate knowledge about the physical configuration of something, including their respective minds, they have to agree about that configuration, regardless of that they possibly have different values.
(2): I’m personally a bit confused about Eliezer’s constant promotion of a language that de-subjectivizes morality. In most debates “objective” and “subjective” may entail a confusion when viewed in a naturalistic light; however, as I understand Eliezer’s stance does boil down to a traditionally subjective viewpoint in the sense that it opposes the religious notion of morality as light shining down from the skies (and the notion of universally compelling arguments).
In regards to infallibility, an agent at most times has imperfect knowledge of right; I can’t see how subjectivity entails infallibility. I don’t even have perfect access to my current values, and there is also a huge set of moral arguments that would compel me to modify my current values if I heard them.
(3) The “why right means promoting X and Y” question is addressed by a recursive justification as discussed here and very specifically in the last paragraphs of Meaning of Right. If I ask “why should I do what is right?”, that roughly means “why should I do what I should do?” or “why is right what is right?”. I happen to be a mind that is compelled by a certain class of moral arguments, and I can reflect on this fact using my current mind, and, naturally, find that I’m compelled by a certain class of moral arguments.
re: infallibility—right, the objection is not that you could infallibly know that XYZ is right. Rather, the problem is that you could infallibly know that your fundamental values are right (though you might not know what your fundamental values are).
Rephrased, this knowledge is just the notion that you instantiate some computation instead of not doing (or being) anything. This way, my confidence in its truth is very high, although of course not 1.
We know we instantiate some computation. But it’s a pre-theoretic datum that we don’t know that our fundamental values are right. So EY’s theory misdescribes the concept of rightness.
(This is basically a variation on Moore’s Open Question Argument.)
I’d be okay with a strong AI that correctly followed my values, regardless of whether they’re “right” by any other criterion.
If you think you wouldn’t be okay with such an AI, I suspect the most likely explanation is that you’re confused about the concept of “your values”. Namely, if you yearn to discover some simple external formula like the categorical imperative and then enact the outcomes prescribed by that formula, then that’s just another fact about your personal makeup that has to be taken into account by the AI.
And if you agree that you would be okay with such an AI, that means Eliezer’s metaethics is adequate for its stated goal (creating friendly AI), whatever other theoretical drawbacks it might have.
(1): I think it’s a prominent naturalistic feature; as EY said above, in a physical universe there are only quantum amplitudes, and if two agents have sufficiently accurate knowledge about the physical configuration of something, including their respective minds, they have to agree about that configuration, regardless of that they possibly have different values.
(2): I’m personally a bit confused about Eliezer’s constant promotion of a language that de-subjectivizes morality. In most debates “objective” and “subjective” may entail a confusion when viewed in a naturalistic light; however, as I understand Eliezer’s stance does boil down to a traditionally subjective viewpoint in the sense that it opposes the religious notion of morality as light shining down from the skies (and the notion of universally compelling arguments).
In regards to infallibility, an agent at most times has imperfect knowledge of right; I can’t see how subjectivity entails infallibility. I don’t even have perfect access to my current values, and there is also a huge set of moral arguments that would compel me to modify my current values if I heard them.
(3) The “why right means promoting X and Y” question is addressed by a recursive justification as discussed here and very specifically in the last paragraphs of Meaning of Right. If I ask “why should I do what is right?”, that roughly means “why should I do what I should do?” or “why is right what is right?”. I happen to be a mind that is compelled by a certain class of moral arguments, and I can reflect on this fact using my current mind, and, naturally, find that I’m compelled by a certain class of moral arguments.
EDIT: see also komponisto’s comment.
re: infallibility—right, the objection is not that you could infallibly know that XYZ is right. Rather, the problem is that you could infallibly know that your fundamental values are right (though you might not know what your fundamental values are).
Rephrased, this knowledge is just the notion that you instantiate some computation instead of not doing (or being) anything. This way, my confidence in its truth is very high, although of course not 1.
We know we instantiate some computation. But it’s a pre-theoretic datum that we don’t know that our fundamental values are right. So EY’s theory misdescribes the concept of rightness.
(This is basically a variation on Moore’s Open Question Argument.)
Huh?
I’d be okay with a strong AI that correctly followed my values, regardless of whether they’re “right” by any other criterion.
If you think you wouldn’t be okay with such an AI, I suspect the most likely explanation is that you’re confused about the concept of “your values”. Namely, if you yearn to discover some simple external formula like the categorical imperative and then enact the outcomes prescribed by that formula, then that’s just another fact about your personal makeup that has to be taken into account by the AI.
And if you agree that you would be okay with such an AI, that means Eliezer’s metaethics is adequate for its stated goal (creating friendly AI), whatever other theoretical drawbacks it might have.