It seems like “human values” aren’t particularly reflective then? Like I could describe the behavioral properties of a species of animal, including what they value or don’t value.
A lot of the particulars of humans’ values are heavily reflective. Two examples:
A large chunk of humans’ terminal values involves their emotional/experience states—happy, sad, in pain, delighted, etc.
Humans typically want ~terminally to have some control over their own futures.
Contrast that to e.g. a blue-minimizing robot, which just tries to minimize the amount of blue stuff in the universe. That utility function involves reflection only insofar as the robot is (or isn’t) blue.
I think you can unroll any of the positive examples by references to facts about the speaker. To be honest, I don’t understand what is supposed to be so reflective about “actual human values”, but perhaps it’s that the ontology is defined with reference to fairly detailed empirical facts about humans.
It seems like “human values” aren’t particularly reflective then? Like I could describe the behavioral properties of a species of animal, including what they value or don’t value.
But that leaves something out?
Decomposing policy into values and value-neutral part seems to be hard and I suspect actual solution requires reflection.
A lot of the particulars of humans’ values are heavily reflective. Two examples:
A large chunk of humans’ terminal values involves their emotional/experience states—happy, sad, in pain, delighted, etc.
Humans typically want ~terminally to have some control over their own futures.
Contrast that to e.g. a blue-minimizing robot, which just tries to minimize the amount of blue stuff in the universe. That utility function involves reflection only insofar as the robot is (or isn’t) blue.
I think you can unroll any of the positive examples by references to facts about the speaker. To be honest, I don’t understand what is supposed to be so reflective about “actual human values”, but perhaps it’s that the ontology is defined with reference to fairly detailed empirical facts about humans.