Some of these (e.g. Roko’s) concerns might be clarified in terms of the distinctions between sense, reference, and reference-fixing descriptions. I take it Eliezer wants to use ‘right’ as a rigid designator to denote some particular set of terminal values, but others have pointed out that this reference fact is fixed by means of a seemingly ‘relative’ procedure (namely, whatever terminal values he himself happens to hold, on some appropriate [if somewhat mysterious] idealization). There is also some concern that this doesn’t match the plain meaning or sense of the term ‘right’, as everyone else understands it.
Some of these (e.g. Roko’s) concerns might be clarified in terms of the distinctions between sense, reference, and reference-fixing descriptions. I take it Eliezer wants to use ‘right’ as a rigid designator to denote some particular set of terminal values, but others have pointed out that this reference fact is fixed by means of a seemingly ‘relative’ procedure (namely, whatever terminal values he himself happens to hold, on some appropriate [if somewhat mysterious] idealization). There is also some concern that this doesn’t match the plain meaning or sense of the term ‘right’, as everyone else understands it.