It’s not as if average-utilitarianism is the only possible answer. Real life today already allows for subcultures who enjoy diverging from most of humanity. Any goal system halfway worth implementing would also allow for such.
According to us. How certain are you that the CEV of all of humanity agrees?
The fact that they exist today isn’t an answer; it could be (and to some degree is) because eradicating them would be too costly, morally or economically.
How certain are you that the CEV of all of humanity agrees?
Since CEV is, AFAICT, defined by the mean of all humans’ utility functions (derived from reflective equilibrium), it disagrees by definition. But CEV is not divine revelation: it’s just the best Eliezer could do at the time. As we learn more about the evaluative and social cognition underlying our “moral” judgements, I expect to be able to dissolve a priori philosophizing and build a better Friendly goal-system than CEV.
Of course, this is because I don’t believe in normative ethics in the normal sense (post-hoc consequentialist based on Peter Railton’s style of moral realism, blah blah blah), so I’m sure this will wind up a massive debate at some point. I strongly doubt the armchair philosophers will go down without a fight.
My guess is that Eliezer will be horrified at the results of CEV—despite the fact that most people will be happy with it.
This is obvious given the degree to which Eliezer’s personal morality diverges from the morality of the human race.
He is not the only one who’d be horrified. Median humanity scares me.
Would it be fair to ignore them and make a geek/LW-specific CEV?
No, but I’m not sure how much I care.
It’s not as if average-utilitarianism is the only possible answer. Real life today already allows for subcultures who enjoy diverging from most of humanity. Any goal system halfway worth implementing would also allow for such.
According to us. How certain are you that the CEV of all of humanity agrees?
The fact that they exist today isn’t an answer; it could be (and to some degree is) because eradicating them would be too costly, morally or economically.
Since CEV is, AFAICT, defined by the mean of all humans’ utility functions (derived from reflective equilibrium), it disagrees by definition. But CEV is not divine revelation: it’s just the best Eliezer could do at the time. As we learn more about the evaluative and social cognition underlying our “moral” judgements, I expect to be able to dissolve a priori philosophizing and build a better Friendly goal-system than CEV.
Of course, this is because I don’t believe in normative ethics in the normal sense (post-hoc consequentialist based on Peter Railton’s style of moral realism, blah blah blah), so I’m sure this will wind up a massive debate at some point. I strongly doubt the armchair philosophers will go down without a fight.