“Mapping human values” strikes me as being just as productive an activity for AGI development as mapping constellations is for astronomy: it is superficial and the really important stuff is much deeper. Sample questions: Where do values come from? How does one simulate a value for an optimizer? Is value even the right level of thinking or is it a byproduct of something more fundamental?
Didn’t most astronomical progress follow and depend upon the mapping of constellations? Even when it turned out that there wasn’t much more than random chance to the patterns of stars, precisely mapping those patterns let us compute parallaxes, record and eventually predict planetary motion, etc. Biology and chemistry started with similar tedious “cataloging” activities long before theories developed to unify all that data… and now this is starting to feel like less of an analogy and less of a coincidence: if you want a good theory to explain a complicated phenomenon, don’t you almost have to start by accumulating a lot of data?
Mapping stars and especially mapping planets turned out to be really important for the development of astronomy. Constellations turn out to be a useless concept. Asking lots of people what constellations they see or where they think the boundaries are would have been wasted astronomical effort.
To return to the real topic under discussion: It might be the case that values are useless and we should only talk about preferences, or somesuch. I am agnostic on this point; I wanted to give an example of how some concept might turn out to be not worth collecting empirical data on.
“Mapping human values” strikes me as being just as productive an activity for AGI development as mapping constellations is for astronomy: it is superficial and the really important stuff is much deeper. Sample questions: Where do values come from? How does one simulate a value for an optimizer? Is value even the right level of thinking or is it a byproduct of something more fundamental?
Didn’t most astronomical progress follow and depend upon the mapping of constellations? Even when it turned out that there wasn’t much more than random chance to the patterns of stars, precisely mapping those patterns let us compute parallaxes, record and eventually predict planetary motion, etc. Biology and chemistry started with similar tedious “cataloging” activities long before theories developed to unify all that data… and now this is starting to feel like less of an analogy and less of a coincidence: if you want a good theory to explain a complicated phenomenon, don’t you almost have to start by accumulating a lot of data?
Mapping stars and especially mapping planets turned out to be really important for the development of astronomy. Constellations turn out to be a useless concept. Asking lots of people what constellations they see or where they think the boundaries are would have been wasted astronomical effort.
To return to the real topic under discussion: It might be the case that values are useless and we should only talk about preferences, or somesuch. I am agnostic on this point; I wanted to give an example of how some concept might turn out to be not worth collecting empirical data on.