Eliezer’s justification for extending the “complexity of evolved motivation” to the “complexity of reflective equilibrium value”, as far as I can tell, is that he knows or strongly suspects that the relevant reflective equilibrium involves people leading good lives, with the word “good” to be filled in by fun theory along the lines of what he’s posted in the fun theory sequence, which in turn refers back to complex evolved motivation. That seems plausible enough to me, but I agree the aforementioned extension isn’t an automatic step.
I think it would be reasonable if Eliezer thinks his own reflective equilibrium involves people leading such “good” lives, but not if he thinks everyone has similar reflective equilibria. I know he draws heavily on the notion of psychological unity of humankind, but it seems insufficient to ensure that people share the same “reflective equilibrium value” as opposed to “evolved motivation”, given that people start off with different moral intuitions and the process of reflecting on those intuitions seems more divergent than convergent. Again, for people who already have a strong intuition that morality or their values should be simple, it seems quite plausible that they would end up with “reflective equilibrium value” that is quite simple compared to “Godshatter”.
people who already have a strong intuition that morality or their values should be simple
Such an intuition may not survive 1) a better intuitive understanding of how human psychology can make some things seem simple even when they are not, and 2) a better intuitive understanding of how all the usual arguments for Occam’s razor apply to facts but not to values.
Eliezer’s justification for extending the “complexity of evolved motivation” to the “complexity of reflective equilibrium value”, as far as I can tell, is that he knows or strongly suspects that the relevant reflective equilibrium involves people leading good lives, with the word “good” to be filled in by fun theory along the lines of what he’s posted in the fun theory sequence, which in turn refers back to complex evolved motivation. That seems plausible enough to me, but I agree the aforementioned extension isn’t an automatic step.
I think it would be reasonable if Eliezer thinks his own reflective equilibrium involves people leading such “good” lives, but not if he thinks everyone has similar reflective equilibria. I know he draws heavily on the notion of psychological unity of humankind, but it seems insufficient to ensure that people share the same “reflective equilibrium value” as opposed to “evolved motivation”, given that people start off with different moral intuitions and the process of reflecting on those intuitions seems more divergent than convergent. Again, for people who already have a strong intuition that morality or their values should be simple, it seems quite plausible that they would end up with “reflective equilibrium value” that is quite simple compared to “Godshatter”.
Such an intuition may not survive 1) a better intuitive understanding of how human psychology can make some things seem simple even when they are not, and 2) a better intuitive understanding of how all the usual arguments for Occam’s razor apply to facts but not to values.
Why do you believe this? What sorts of examples do you have in mind?
See http://lesswrong.com/r/discussion/lw/49j/why_do_we_engage_in_moral_simplification/