If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.
I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/mental state”, then why would the things humans actually value necessarily be the same as Goodness?
A common use of “Human Values” is in sentences like “we should align AI with Human Values” or “it would be good to maximize Human Values upon reflection”, i.e. normative claims about how Human Values are good and should be achieved. However, if you’re not a moral realist, there’s no (or very little) reason to believe that humans, even if they reflect for a long time etc., will arrive on the same values. Most of the time if someone says “Human Values” they don’t mean to include the values of Hitler or a serial killer. This makes the term confusing, because it can both be used descriptively and normatively, and the normative use is common enough to make it confusing when used as a purely descriptive term.
I agree that if you’re a moral realist, it’s useful to have a term for “preferences shared amongst most humans” as distinct from Goodness, but Human Values is a bad choice because:
It implies preferences are more consistent amongst humans than they really are
The use of “Human Values” has been too polluted by others using it in a normative sense
I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/mental state”, then why would the things humans actually value necessarily be the same as Goodness?
A common use of “Human Values” is in sentences like “we should align AI with Human Values” or “it would be good to maximize Human Values upon reflection”, i.e. normative claims about how Human Values are good and should be achieved. However, if you’re not a moral realist, there’s no (or very little) reason to believe that humans, even if they reflect for a long time etc., will arrive on the same values. Most of the time if someone says “Human Values” they don’t mean to include the values of Hitler or a serial killer. This makes the term confusing, because it can both be used descriptively and normatively, and the normative use is common enough to make it confusing when used as a purely descriptive term.
I agree that if you’re a moral realist, it’s useful to have a term for “preferences shared amongst most humans” as distinct from Goodness, but Human Values is a bad choice because:
It implies preferences are more consistent amongst humans than they really are
The use of “Human Values” has been too polluted by others using it in a normative sense