I feel like reflective stability is what caused me to reject utility. Specifically, it seems like it is impossible to be reflectively stable if I am the kind of mind that would follow the style of argument given for the independence axiom. It seems like there is a conflict between reflective stability and Bayesian updating.
I am choosing reflective stability, in spite of the fact that loosing updating is making things very messy and confusing (especially in the logical setting), because reflective stability is that important.
When I lose updating, the independence axiom, and thus utility goes along with it.
UDT still has utility functions, even though it doesn’t have independence… Is it just a terminological issue? Like you want to call the representation of value in whatever the correct decision theory turns out to be something besides “utility”? If so, why?
But where does UDT get those utility functions from, why does it care about expected utility specifically and not arbitrary preference over policies? Utility functions seem to centrally originate from updateful agents, which take many actions in many hypothetical situations, coherent with each other, forcing preference to be describable as expected utility. Such agents can then become reflectively stable by turning to UDT, now only ever taking a single decision about policy, in the single situation of total ignorance, with nothing else for it to be coherent with.
So by becoming updateless, a UDT agent loses contact with the origin of (motivation for) its own utility function. To keep it, it would still implicitly need an updateful point of view, with its many situations that consitutute the affordance for acting coherently, to motivate its preference to have the specific form of expected utility. Otherwise it only has the one situation, and its preference and policy could be anything, with no opportunity to be constrained by coherence.
I think UDT as you specified it has utility functions. What do you mean by doesn’t have independence? I am advocating for an updateless agent model that might strictly prefer a mixture between outcomes A and B to either A or B deterministically. I think an agent model with this property should not be described as having a “utility.” Maybe I am conflating “utility” with expected utility maximization/VNM and you are meaning something more general?
If you mean by utility something more general than utility as used in EUM, then I think it is mostly a terminological issue.
I think I endorse the word “utility” without any qualifiers as referring to EUM. In part because I think that is how it is used, and in part because EUM is nice enough to deserve the word utility.
Even if EUM doesn’t get “utility”, I think it at least gets “utility function”, since “function” implies cardinal utility rather than ordinal utility and I think people almost always mean EUM when talking about cardinal utility.
I personally care about cardinal utility, where the magnitude of the utility is information about how to aggregate rather than information about how to take lotteries, but I think this is a very small minority usage of cardinal utility, so I don’t think it should change the naming convention very much.
I feel like reflective stability is what caused me to reject utility. Specifically, it seems like it is impossible to be reflectively stable if I am the kind of mind that would follow the style of argument given for the independence axiom. It seems like there is a conflict between reflective stability and Bayesian updating.
I am choosing reflective stability, in spite of the fact that loosing updating is making things very messy and confusing (especially in the logical setting), because reflective stability is that important.
When I lose updating, the independence axiom, and thus utility goes along with it.
Although I note that my flavor of rejecting utility functions is trying to replace them with something more general, not something incompatible.
UDT still has utility functions, even though it doesn’t have independence… Is it just a terminological issue? Like you want to call the representation of value in whatever the correct decision theory turns out to be something besides “utility”? If so, why?
But where does UDT get those utility functions from, why does it care about expected utility specifically and not arbitrary preference over policies? Utility functions seem to centrally originate from updateful agents, which take many actions in many hypothetical situations, coherent with each other, forcing preference to be describable as expected utility. Such agents can then become reflectively stable by turning to UDT, now only ever taking a single decision about policy, in the single situation of total ignorance, with nothing else for it to be coherent with.
So by becoming updateless, a UDT agent loses contact with the origin of (motivation for) its own utility function. To keep it, it would still implicitly need an updateful point of view, with its many situations that consitutute the affordance for acting coherently, to motivate its preference to have the specific form of expected utility. Otherwise it only has the one situation, and its preference and policy could be anything, with no opportunity to be constrained by coherence.
I think UDT as you specified it has utility functions. What do you mean by doesn’t have independence? I am advocating for an updateless agent model that might strictly prefer a mixture between outcomes A and B to either A or B deterministically. I think an agent model with this property should not be described as having a “utility.” Maybe I am conflating “utility” with expected utility maximization/VNM and you are meaning something more general?
If you mean by utility something more general than utility as used in EUM, then I think it is mostly a terminological issue.
I think I endorse the word “utility” without any qualifiers as referring to EUM. In part because I think that is how it is used, and in part because EUM is nice enough to deserve the word utility.
Even if EUM doesn’t get “utility”, I think it at least gets “utility function”, since “function” implies cardinal utility rather than ordinal utility and I think people almost always mean EUM when talking about cardinal utility.
I personally care about cardinal utility, where the magnitude of the utility is information about how to aggregate rather than information about how to take lotteries, but I think this is a very small minority usage of cardinal utility, so I don’t think it should change the naming convention very much.