I vote for the first possibility—that utility functions are not particularly good match for human preferences, for following reasons:
1) I have never seen one, at least valid outside very narrow subject matter. That implies that people are not good at drawing these functions, which may be caused by the fact that these functions could in reality be very complicated, if even they exist. So even if my preferences are consistent with some utility function, any practical application would apply some strongly simplified model of the function, which could differ significantly from my real preferences.
2) As Roko has said, the utility function is defined rather on histories of universe than on its states. Since my preferences change (and I don’t have general desire to keep all of them constant), I am not sure how to treat the time entanglement.
3) From a purely practical point of view, assigning numerical values to very improbable possibilities is prone to numerical errors.
For me some mix of deontology with utilitarianism works better than pure utilitarianism as real-life ethical and decision-making theory.
I vote for the first possibility—that utility functions are not particularly good match for human preferences, for following reasons: 1) I have never seen one, at least valid outside very narrow subject matter. That implies that people are not good at drawing these functions, which may be caused by the fact that these functions could in reality be very complicated, if even they exist. So even if my preferences are consistent with some utility function, any practical application would apply some strongly simplified model of the function, which could differ significantly from my real preferences. 2) As Roko has said, the utility function is defined rather on histories of universe than on its states. Since my preferences change (and I don’t have general desire to keep all of them constant), I am not sure how to treat the time entanglement. 3) From a purely practical point of view, assigning numerical values to very improbable possibilities is prone to numerical errors.
For me some mix of deontology with utilitarianism works better than pure utilitarianism as real-life ethical and decision-making theory.