Part of what I’ve been trying to do in book reviews such as The Geography of Thought, WEIRDest People, and The Amish has been to illuminate how much of what we think of as value differences are mostly different strategies for achieving some widely shared underlying values such as prosperity, safety, happiness, and life satisfaction.
If a human from a million years ago evaluated us by our policies, then I agree they’d be disappointed. But if they evaluated us by more direct measures of our quality of life, I’d expect them to be rather satisfied. The latter is most of what matters to me.
I don’t like cat burning or religion. But opinions on those topics seem mostly unrelated to what I hope to see in 500 years.
This is a very good point. I’d sorta defend myself by claiming that “what would you do with the galaxy” (and how you rate that) is unusually determined by memetics compared to what you eat for breakfast (and how you rate that). What you eat for breakfast currently has a way bigger impact on your QOL, but it’s more closely tied to supervisory signals shared across humans.
On the one hand, this means I’m picking on a special case, on the other hand, I think that special case is a pretty good analogy for building AI that becomes way more powerful after training.
I disagree.
Part of what I’ve been trying to do in book reviews such as The Geography of Thought, WEIRDest People, and The Amish has been to illuminate how much of what we think of as value differences are mostly different strategies for achieving some widely shared underlying values such as prosperity, safety, happiness, and life satisfaction.
If a human from a million years ago evaluated us by our policies, then I agree they’d be disappointed. But if they evaluated us by more direct measures of our quality of life, I’d expect them to be rather satisfied. The latter is most of what matters to me.
I don’t like cat burning or religion. But opinions on those topics seem mostly unrelated to what I hope to see in 500 years.
This is a very good point. I’d sorta defend myself by claiming that “what would you do with the galaxy” (and how you rate that) is unusually determined by memetics compared to what you eat for breakfast (and how you rate that). What you eat for breakfast currently has a way bigger impact on your QOL, but it’s more closely tied to supervisory signals shared across humans.
On the one hand, this means I’m picking on a special case, on the other hand, I think that special case is a pretty good analogy for building AI that becomes way more powerful after training.