“Utility functions are really bad match for human preferences, and one of the major premises we accept is wrong.”
Given the sheer messiness and mechanical influence involved in human brains, it’s not even clear we have real ‘values’ which could be examined on a utility function, rather than simple dominant-interestedness that happens for largely unconscious and semi-arbitrary reasons.
“Utility functions are really bad match for human preferences, and one of the major premises we accept is wrong.” Given the sheer messiness and mechanical influence involved in human brains, it’s not even clear we have real ‘values’ which could be examined on a utility function, rather than simple dominant-interestedness that happens for largely unconscious and semi-arbitrary reasons.