Because when I say “that big function”, and you say “right”, we are dereferencing two different pointers to the same unverbalizable abstract computation.
No, the other person is dereferencing a pointer to their big function, which may or may not be the same as yours. This is the one place it doesn’t add up to normality: not everyone need have the same function. Eliezer-rightness is objective, a one-place function, but it seems to me the ordinary usage of “right” goes further: it’s assumed that everybody means the same thing by, not just “Eliezer-right”, but “right”. I don’t see how this metamorality allows for that, or how any sensible one could. (Not that it bothers me.)
I believe Elizer is asserting here that “right”=CEV(“Eliezer-right”); it is an extrapolation of “Eliezer-right” to “Eliezer_{perfectly-rational}-right”. And he asserts, with some justification, that CEV(“[Person]-right”)=”right” for *nearly all values of [Person].
EDIT:Obviously [Sociopath] does not fit this. s/all/nearly all/g
The mentions of “neurological damage 1” and “neurological damage 2″ in this comment seem to suggest that EY would consider sadistic sociopaths No True Scotsmen for these purposes.
No, it isn’t. Corrected my above comment to reflect that.
I’d consider sociopathy a degenerate case where morality as we understand it does not have any meaningful role in the decision-making process and never has. Any morality which asks to universalize to all humans, including sociopaths, is likely to be pure preference.
Follow-up question: What is the likelihood that different modal forms of morality are fundamental?
I.e., suppose the dichotomy presented by George Lakoff’s Moral Politics turns out to describe fundamental local maxima in morality-space, which human minds can imperfectly embody.
To math this up a little, suppose the CEV of Morality Attractor 1 computes to “maximize the absolute median QALY”, while the CEV of Morality Attractor 2 computes to “maximize the the amount by which the median QALY of my in-group exceeds the median QALY of all sapiences as a whole”, and neither of those attractors have any particularly universal mathematical reason to favor them. Then, however the FAI searches the Morality domain for a CEV, it is equally likely to settle on some starry-eyed global Uplift as it is to produce nigh-infinite destitute subjects suffering indescribable anguish and despair so that a few Uplifted utility monsters have enough necks to rest their boots on.
And before anyone objects that Morality Attractor 2 is too appalling for anyone to seriously advocate, note that it has been the default behavior of most civilized societies for the majority of human history, so it must have SOMETHING going for it. Maybe Morality Attractor 1 just seems more accessible because it’s the one advocated by the mother culture that raised us, not because it’s actually what most humans tend towards as IQ/g/whatever approaches infinity.
First, I think you are significantly off-base in your contention that Morality Attractor 2 has been implemented in most civilized societies. What you see as Attractor 2 I think is better explained by Attractor 2a: “Maximize the absolute median QALY of the in-group.”, and that Clever Arguers throughout history have appealed to this desire by pointing out how much better off the in-group was, compared to a specific acceptable-target outgroup. It is, after all, much easier to provide a relative QALY surplus than an absolute QALY surplus, and our corrupted hardware is not very good at distinguishing the two. As anecdotal evidence, I consider my own morality significantly closer to 2a than 1, but definitely not similar to 2.
I would further say that it seems unlikely that the basic moral impulse is actually restricted to an arbitrary ingroup. One of those ‘perfect information’ aspects inherent in defining the output of the CEV would be knowing the life story of every person on the planet. Which is, if my knowledge of psychology is correct, basically an express ticket into the moral ingroup. This is why the single-child quarter-donation signs work, when appeals to the huge number of children suffering from don’t.
So overall, I don’t find that suggestion plausible. Someone with human-typical psychology who knew every person in existence as well as we know our friends, which is basically the postulated mind whose utility function is the CEV, would inherently value all their QALY.
Bravo. But:
No, the other person is dereferencing a pointer to their big function, which may or may not be the same as yours. This is the one place it doesn’t add up to normality: not everyone need have the same function. Eliezer-rightness is objective, a one-place function, but it seems to me the ordinary usage of “right” goes further: it’s assumed that everybody means the same thing by, not just “Eliezer-right”, but “right”. I don’t see how this metamorality allows for that, or how any sensible one could. (Not that it bothers me.)
I believe Elizer is asserting here that “right”=CEV(“Eliezer-right”); it is an extrapolation of “Eliezer-right” to “Eliezer_{perfectly-rational}-right”. And he asserts, with some justification, that CEV(“[Person]-right”)=”right” for *nearly all values of [Person].
EDIT:Obviously [Sociopath] does not fit this. s/all/nearly all/g
Naive question: by these definitions, [sadistic sociopath] ⊆ [Person]?
The mentions of “neurological damage 1” and “neurological damage 2″ in this comment seem to suggest that EY would consider sadistic sociopaths No True Scotsmen for these purposes.
No, it isn’t. Corrected my above comment to reflect that.
I’d consider sociopathy a degenerate case where morality as we understand it does not have any meaningful role in the decision-making process and never has. Any morality which asks to universalize to all humans, including sociopaths, is likely to be pure preference.
nod
Follow-up question: What is the likelihood that different modal forms of morality are fundamental?
I.e., suppose the dichotomy presented by George Lakoff’s Moral Politics turns out to describe fundamental local maxima in morality-space, which human minds can imperfectly embody.
To math this up a little, suppose the CEV of Morality Attractor 1 computes to “maximize the absolute median QALY”, while the CEV of Morality Attractor 2 computes to “maximize the the amount by which the median QALY of my in-group exceeds the median QALY of all sapiences as a whole”, and neither of those attractors have any particularly universal mathematical reason to favor them. Then, however the FAI searches the Morality domain for a CEV, it is equally likely to settle on some starry-eyed global Uplift as it is to produce nigh-infinite destitute subjects suffering indescribable anguish and despair so that a few Uplifted utility monsters have enough necks to rest their boots on.
And before anyone objects that Morality Attractor 2 is too appalling for anyone to seriously advocate, note that it has been the default behavior of most civilized societies for the majority of human history, so it must have SOMETHING going for it. Maybe Morality Attractor 1 just seems more accessible because it’s the one advocated by the mother culture that raised us, not because it’s actually what most humans tend towards as IQ/g/whatever approaches infinity.
First, I think you are significantly off-base in your contention that Morality Attractor 2 has been implemented in most civilized societies. What you see as Attractor 2 I think is better explained by Attractor 2a: “Maximize the absolute median QALY of the in-group.”, and that Clever Arguers throughout history have appealed to this desire by pointing out how much better off the in-group was, compared to a specific acceptable-target outgroup. It is, after all, much easier to provide a relative QALY surplus than an absolute QALY surplus, and our corrupted hardware is not very good at distinguishing the two. As anecdotal evidence, I consider my own morality significantly closer to 2a than 1, but definitely not similar to 2.
I would further say that it seems unlikely that the basic moral impulse is actually restricted to an arbitrary ingroup. One of those ‘perfect information’ aspects inherent in defining the output of the CEV would be knowing the life story of every person on the planet. Which is, if my knowledge of psychology is correct, basically an express ticket into the moral ingroup. This is why the single-child quarter-donation signs work, when appeals to the huge number of children suffering from don’t.
So overall, I don’t find that suggestion plausible. Someone with human-typical psychology who knew every person in existence as well as we know our friends, which is basically the postulated mind whose utility function is the CEV, would inherently value all their QALY.