I think the confusion here is that “Goodness” means different things depending on whether you’re a moral realist or anti-realist.
If you’re a moral realist, Goodness is an objective quality that doesn’t depend on your feelings/mental state. What is Good may or may not overlap with what you like/prefer/find yummy, but it doesn’t have to.
If you’re a moral anti-realist, either:
“Goodness” is meaningless.
“Goodness” is a shorthand for something like:
“My fundamental, least changeable preferences/likes/wants”
“The subset of my preferences/likes/wants that many other people share”
“The subset of my preferences/likes/wants that it’s socially acceptable to talk a lot about/encourage others to adopt”
“The subset of my preferences/likes/wants that I want others to adopt”
I think “Human Values” is a very poor phrase because:
If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.
If you’re a moral anti-realist, you can just talk about your preferences, or a particular subset of your preferences (e.g. any of the options listed above).
Instead, people referring to “Human Values” obscure whether they are moral realists or anti-realists, which causes a lot of confusion when determining the implications and logical consistency of their views.
If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.
I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/mental state”, then why would the things humans actually value necessarily be the same as Goodness?
A common use of “Human Values” is in sentences like “we should align AI with Human Values” or “it would be good to maximize Human Values upon reflection”, i.e. normative claims about how Human Values are good and should be achieved. However, if you’re not a moral realist, there’s no (or very little) reason to believe that humans, even if they reflect for a long time etc., will arrive on the same values. Most of the time if someone says “Human Values” they don’t mean to include the values of Hitler or a serial killer. This makes the term confusing, because it can both be used descriptively and normatively, and the normative use is common enough to make it confusing when used as a purely descriptive term.
I agree that if you’re a moral realist, it’s useful to have a term for “preferences shared amongst most humans” as distinct from Goodness, but Human Values is a bad choice because:
It implies preferences are more consistent amongst humans than they really are
The use of “Human Values” has been too polluted by others using it in a normative sense
I think the confusion here is that “Goodness” means different things depending on whether you’re a moral realist or anti-realist.
If you’re a moral realist, Goodness is an objective quality that doesn’t depend on your feelings/mental state. What is Good may or may not overlap with what you like/prefer/find yummy, but it doesn’t have to.
If you’re a moral anti-realist, either:
“Goodness” is meaningless.
“Goodness” is a shorthand for something like:
“My fundamental, least changeable preferences/likes/wants”
“The subset of my preferences/likes/wants that many other people share”
“The subset of my preferences/likes/wants that it’s socially acceptable to talk a lot about/encourage others to adopt”
“The subset of my preferences/likes/wants that I want others to adopt”
I think “Human Values” is a very poor phrase because:
If you’re a moral realist, you can just say “Goodness” instead of “Human Values”.
If you’re a moral anti-realist, you can just talk about your preferences, or a particular subset of your preferences (e.g. any of the options listed above).
Instead, people referring to “Human Values” obscure whether they are moral realists or anti-realists, which causes a lot of confusion when determining the implications and logical consistency of their views.
I notice I am confused. If “Goodness is an objective quality that doesn’t depend on your feelings/mental state”, then why would the things humans actually value necessarily be the same as Goodness?
A common use of “Human Values” is in sentences like “we should align AI with Human Values” or “it would be good to maximize Human Values upon reflection”, i.e. normative claims about how Human Values are good and should be achieved. However, if you’re not a moral realist, there’s no (or very little) reason to believe that humans, even if they reflect for a long time etc., will arrive on the same values. Most of the time if someone says “Human Values” they don’t mean to include the values of Hitler or a serial killer. This makes the term confusing, because it can both be used descriptively and normatively, and the normative use is common enough to make it confusing when used as a purely descriptive term.
I agree that if you’re a moral realist, it’s useful to have a term for “preferences shared amongst most humans” as distinct from Goodness, but Human Values is a bad choice because:
It implies preferences are more consistent amongst humans than they really are
The use of “Human Values” has been too polluted by others using it in a normative sense