As Richard Kennaway noted, it seems considerations about time are muddling things here. If we wanted to be super proper, then preferences should have as objects maximally specific ways the world could be, including the whole history and future of the universe, down to the last detail. Decision theory involving anything more coarse-grained than that is just a useful approximation—e.g. I might have a decision problem with only two outcomes being “You get $10” and “You lose $5,” but we would just be pretending these are the only two ways the world can end up for practical purposes, which is a permissible simplification since in my actual circumstances my desire to have more money over less is independent of other considerations. In particular the objects of preference are not states of the world at a time, since it is not a rational requirement that just because I prefer that one state instead of another obtain at t1 I must also prefer likewise for t2. So, our agent who is perfectly content with their eternal A-B-C cycle is just one who likes this strange history of the world they can bring about, who nevertheless has potentially acyclic preferences. Indeed, this agent is still rational on orthodox decision theory even if they’re paying for every such cycle. Similarly, in your case of the agent who goes through B to get to A, we would want to model the objects of preference as things like “Stay in the A state indefinitely” and “Be in a B-state for a little while and then be in a C-state,” and we would not want A, B, and C themselves to be the objects of preference.
So what does this say about the money-pump argument for acyclicality, which seems to treat preferences as over states-at-a-time? I think the sorts of considerations you raise do show that such arguments don’t go through unless we make certain (perhaps realistic) assumptions about the structure of agents’ preferences, assuming we view the force of the money-pump arguments to come from the possibility of actual exploitation (some proponents don’t care so much about actual exploitability, but think the hypothetical exploitability is indicative of an underlying rational inconsistency). For example, if I have the preferences A≺B≺C≺A, the argument assumes that there is some slightly worse version of A that I still prefer to C, that my preferences are coarse-grained enough that we can speak of my “trading” one of these states of affairs for another, that they are coarse-grained enough that my past trades don’t influence whether I now prefer to trade C for the slightly worse version of A, and so on. If the relevant assumptions are met, then the money-pump argument shows that my cyclic preferences leave me with an outcome that I really will disprefer to having just stuck with A, as opposed to ending up in a state I will later pay to transition out of (which is no fault of my rationality). You’re right to point out that these really are substantive assumptions.
So, in sum: I think (i) the formalism for decision theory is alright in not taking into account the possibility of transitions between outcomes, and only taking into account preferences between outcomes and which outcomes you can affect in a given decision problem; and (ii) formalism aside, your observations do show limitations to/assumptions behind the money-pump arguments insofar as those arguments are supposed to be merely pragmatic. I do not know the extent to which I disagree with you, but I hope this clarifies things.
If we wanted to be super proper, then preferences should have as objects maximally specific ways the world could be, including the whole history and future of the universe, down to the last detail.
[...] it is not a rational requirement that just because I prefer that one state instead of another obtain at t1 I must also prefer likewise for t2.
As you note, you’re not the first commenter here to make these claims. But I just… really don’t buy it?
The main reason is that if you take this view, several famous and popular arguments in decision theory become complete nonsense. The two cases that come to mind immediately are money pumps for non-transitive preferences and diachronic Dutch books. In both cases, the argument assumes an agent that makes multiple successive decisions on the basis of preferences which we assume do not vary with time (or which only vary with time in some specified manner, rather than being totally unconstrained as your proposal would have it).
If sequential decisions like this were simply not the object of study in decision theory, then everyone would immediately reject these arguments by saying “you’ve misunderstood the terminology, you’re making a category error, this is just not what the theory is about.” Indeed, in this case, the arguments probably would not have ever been put forth in the first place. But in reality, these are famous and well-regarded arguments that many people take seriously.
So either there’s a clear and consistent object of study which is not these “maximally specific histories”… or decision theorists are inconsistent / confused about what the object of study is, sometimes claiming it is “maximally specific histories” and sometimes claiming otherwise.
Separately, if the theory really is about “maximally specific histories,” then the theory seems completely irrelevant to real decision-making. Outside of sci-fi or thought experiments, I will never face a “decision problem” in which both of the options are complete specifications of the way the rest of my life might go, all the way up to the moment I die.
And even if that were the sort of thing that could happen, I would find such a decision very hard to make, because I don’t find myself in possession of immediately-accessible preferences about complete life-histories; the only way I could make the decision in practice would be to “calculate” my implicit preferences about the life-histories by aggregating over my preferences about all the individual events that happen within them at particular times, i.e. the kinds of preferences that I actually use to make real-life decisions, and which I “actually have” in the sense that I immediately feel a sense of preferring one thing to another when I compare the two options.
Relatedly, it seems like very little is really “rationally required” for preferences about “maximally specific histories,” so the theory of them does not have much interesting content. We can’t make pragmatic/prudential arguments like money pumps, nor can we fall back on some set of previously held intuitions about what rational decision-making is (because “rational decision-making” in practice always occurs over time and never involves these unnatural choices between complete histories). There’s no clear reason to require the preferences to actually relate to the content of the histories in some “continuous” way; all the “physical” content of the histories, here, is bundled inside contentless labels like “A” and is thus invisible to the theory. I could have a totally nonsensical set of preferences that looks totally different at every moment in time, and yet at this level of resolution, there’s nothing wrong with it (because we can’t see the “moments” at this level, they’re bundled inside objects like “A”). So while we can picture things this way if we feel like it, there’s not much that’s worth saying about this picture of things, nor is there much that it can tell us about reality. At this point we’re basically just saying “there a set with some antisymmetric binary relation on it, and maybe we’ll have an argument about whether the relation ‘ought’ to be transitive.” There’s no philosophy left here, only a bit of dull, trivial, and unmotivated mathematics.
So, in my experience it’s common for decision theorists in philosophy to take preferences to be over possible worlds and probability distributions over such (the specification of which includes the past and future), and when coarse-graining they take outcomes to be sets of possible worlds. (What most philosophers do is, of course, irrelevant to the matter of how it’s best to do things, but I just want to separate “my proposal” from what I (perhaps mistakenly) take to be common.) As you say, no agent remotely close to actual agents will have preferences where details down to the location of every particle in 10,000 BC make a difference, which is why we coarse-grain. But this fact doesn’t mean that maximally-specific worlds or sets of such shouldn’t be what are modeled as outcomes, as opposed to outcomes being ways the world can be at specific times (which I take to be your understanding of things; correct me if I’m wrong). Rather, it just means a lot of these possible worlds will end up getting the same utility because the agent is indifferent between them, and so we’ll be able to treat them as the same outcome.
I worry I might be misunderstanding your point in the last few paragraphs, as I don’t understand how having maximally-specific worlds as outcomes is counter to our commonsense thinking of decision-making. It is true I don’t have a felt preference between “I get $100 and the universe was in state S five billion years ago” and “I get $100 and the universe was in state T five billion years ago,” let alone anything more specific. But that is just to say I am indifferent—I weakly prefer either outcome to the other, and this fact is as intuitively accessible to me as the fact that I prefer getting $100 to getting $10. As far as I can tell, the only consequence of making things too fine-grained is that you include irrelevant details to which the agent is indifferent and you make the model needlessly unwieldy—not that it invokes a concept of preference divorced from actual decision-making.
Outside of sci-fi or thought experiments, I will never face a “decision problem” in which both of the options are complete specifications of the way the rest of my life might go, all the way up to the moment I die.
Every decision problem can be modeled as being between probability distributions over such histories, or over sets of such histories if we want to coarse-grain and ignore minute details to which the agent is indifferent. Now, of course, a real human never has precise probabilities associated with the most minute detail—but the same is true even if the outcomes are states-at-times, and even non-maximally-specific states-at-times. Idealizing that matter away seems to be something any model has to do, and I’m not seeing what problems are introduced specifically by taking outcomes to be histories instead of states-at-times.
The two cases that come to mind immediately are money pumps for non-transitive preferences and diachronic Dutch books. In both cases, the argument assumes an agent that makes multiple successive decisions on the basis of preferences which we assume do not vary with time (or which only vary with time in some specified manner, rather than being totally unconstrained as your proposal would have it).
But having outcomes be sets of maximally-specific world-histories doesn’t prevent us from being able to model sequential decisions. All we need to do so is to assume that the agent can make choices at different times which make a difference as to whether a more-or-less preferred world-history will obtain. For example, say I have a time-dependence preference of preferring to wear a green shirt on weekends and a red shirt on weekdays, so I’m willing to pay $1 to trade shirts every Saturday and Monday. This would not make me a money-pump, rather I’m just someone who’s willing to pay to keep their favorite shirt at the moment on. But allowing time-dependent preferences like this doesn’t prevent money-pump and diachronic Dutch book arguments from ever going through, for I still prefer wearing a red shirt next Monday to wearing a green shirt then, if everything else about the world-history is kept the same. If I in addition prefer wearing a yellow shirt on a weekday to wearing a red one, but then also prefer a green shirt on a weekday to a yellow one, then my preferences are properly cyclic and I will be a money-pump. For let’s say I have a ticket that entitles me to a red shirt next Monday; then suppose I trade that ticket for a yellow shirt ticket, then that one for a green shirt ticket, then I pay $1 to trade that for my original red shirt ticket. Assuming my preferences merely concern (i) what color shirt I wear on what day and (ii) my having more money than less, I will end up in a state that I think is worse than just having made no trade at all. This is so even though my preferences are not merely over shirt-colors and money, but rather coarse-grained sets of world-histories.
All that is needed for the pragmatic money-pump argument to go through is that my preferences are coarse-grained enough that we can speak of my making sequential choices as to which world-histories will obtain, such that facts about what choices I have made in these very cases do not influence the world-history in a way that makes a difference to my preferences. This seems pretty safe in the case of humans, and a result showing transitivity to be rationally required under such assumptions still seems important. Even in the case of a realistic agent who really likes paying to go through the A-B-C-A cycle, we could imagine alternate decision problems where they get to choose whether they get to go through that cycle during a future time-interval, and genuinely exploit them if those preferences are cyclic.
I worry that what I described above in the shirt-color example falls under what you mean by “or which only vary with time in some specified manner, rather than being totally unconstrained as your proposal would have it.” A world-history is a specification of the world-state at each time, similar in kind to “I wear a red shirt next Monday.” As I said before, the model would allow for weird fine-grained preferences over world-histories, but realistic uses of it will have as outcomes larger sets of world-histories like the set of histories where I wear a red shirt next Monday and have $1000, and so on. This is similar to how standard probability theory in principle allows one to talk about probabilities of propositions so specific that no one could ever think or care about them, but for practical purposes we just don’t include those details when we actually apply it.
I’m concerned that I may be losing track of where our disagreement is. So, just to recap: I took your arguments in the OP to be addressed if we understand outcomes (the things to which utilities are assigned) to be world-histories or sets of world-histories, so we can understand the difference between an agent who’s content with paying a dollar for every A-B-C-A cycle and an agent for whom that would be indicative of irrationality. You alleged that if we understand outcomes in this way, and not merely as possible states of the world with no reference to time, then (i) we cannot model sequential decisions, (ii) such a model would have an unrealistic portrayal of decision-making, and (iii) there will be too few constraints on preferences to get anything interesting. I reply: (i) sequential decisions can still be modeled as choices between world-histories, and indeed in an intuitive way if we make the realistic assumption that the agent’s choices in this very sequence by themselves do not make a difference to whether a world-history is preferred; (ii) the decisions do seem to be intuitively modeled if we coarse-grain the sets of world-histories appropriately; and (iii) even if money-pump arguments only go through in the case of agents who have a certain additional structure to their preferences, this additional structure is satisfied by realistic agents and so the restricted results will be important. If I am wrong about what is at issue, feel free to just correct me and ignore anything I said under false assumptions.
If we wanted to be super proper, then preferences should have as objects maximally specific ways the world could be, including the whole history and future of the universe, down to the last detail. Decision theory involving anything more coarse-grained than that is just a useful approximation
Preferences can be equally rigorously defined over events if probabilities and utilities are also available. Call a possible world ω, the set of all possible worlds Ω, and an a set E such that E⊆Ω an “event”. Then the utility U of E is plausibly U(E)=∑ωi∈EP(ωi∣E)U(ωi). This is a probability-weighted average, which derives from dividing the expected utility P(E)U(E)=∑ωi∈EP(ωi)U(ωi) by P(E), to arrive at the formula for U(E) alone.
So if we have both a probability function P and a utility function U over possible worlds, we can also fix a Boolean algebra of events over which those functions are defined. Then a “preference” between two events E≻F is simply U(E)>U(F).
“Events” are a lot more practical than possible worlds, since events don’t have to be maximally specific, and they correspond directly to propositions, which one can “believe” and “desire” to be true. Degrees of belief and degrees of desire can be described by probability and utility functions respectively.
Many propositions can be given event semantics, but with the caveat that events should still be parts of a space of possible worlds, so that “It’s Friday” is not an event (while “When it’s Friday, I try to check the mail” is).
Where do you think is the difference? I agree that there is a problem with indexical content, though this affects both examples. (“It’s (currently) Friday (where I live).”)
Even though it doesn’t solve all problems with indexicals, it’s probably better to not start with possible worlds but instead start with propositions directly, similar to propositional logic. Indeed this is what Richard Jeffrey does. Instead of starting with a set of possible worlds, he starts with a disjunction of two mutually exclusive propositions A and B:
U(A∨B)=P(A)U(A)+P(B)U(B)P(A)+P(B), if P(A∧B)=0,P(A∨B)≠0.
An indexical proposition doesn’t evaluate a possible world (in the sense of a maximally detailed world history), it evaluates a possible world together with a location or time or a subsystem (such as an agent/person) in it. But pointing to some location or subsystem that is already part of the world doesn’t affect the possible world itself, so it makes little sense to have preference that becomes different depending on which location or subsystem we are looking at (from outside the world), while the world remains completely the same. The events that should be objects of preference are made of worlds, not of (world, embedded subsystem) pairs.
Whether “It’s Friday” is not a property of a world, it’s a property of a world together with some admissible spacetime location. You can’t collect a set of possible world histories that centrally have the property of “It’s Friday”. On the other hand, “I try to make sure to check the mail on Fridays” is a property that does distinguish worlds where it’s centrally true (for a specific value of “I”). In general, many observations are indexical, they tell you where you are, which version of you is currently affecting the world, and a policy converts those indexical observations into an actual effect in the world that can be understood as an event, in the sense of a set of possible worlds.
I mean I agree, indexicals don’t really work with interpreting propositions simply as sets of possible worlds, but both sentences contain such indexicals, like “I”, implicitly or explicitly. “I” makes only sense for a specific person at a specific time. “It’s Friday (for me)”, relative to a person and a time, fixes a set of possible worlds where the statement is true. It’s the same for “I try to make sure to check the mail on Fridays”.
After you bundle the values of free variables into the proposition (make a closure), and “I” and such get assigned their specific referents, “It’s Friday” is still in trouble. Because if it gets the current time bundled in it, then it’s either true or false depending on the time, not depending on the world (in the maximally detailed world history sense), and so it’s either true about all worlds or none (“The worlds where it’s Friday on Tuesday”), there is no nontrivial set-of-worlds meaning. But with the mail proposition (or other propositions about policies) there is no problem like that.
I think one issue with the “person+time” context is that we may assume that once I know the time, I must know whether it is Friday or not. A more accurate assessment would be to say that an indexical proposition corresponds to a set of possible worlds together with a person moment, i.e. a complete mental state. The person moment replaces the “person + time” context. This makes it clear that “It’s Friday” is true in some possible worlds and false in others, depending on whether my person moment (my current mental state, including all the evidence I have from perception etc) is spatio-temporally located at a Friday in that possible world. This also makes intuitive sense, since I know my current mental state but that alone is not necessarily sufficient to determine the time of week, and I could be mistaken about whether it’s Friday or not.
A different case is “I am here now” or the classic “I exist”. Which would be true for any person moment and any possible world where that person moment exists. These are “synthetic a priori” propositions. Their truth can be ascertained from introspection alone (“a priori”), but they are “synthetic” rather than “analytic”, since they aren’t true in every possible world, i.e. in worlds were the person moment doesn’t exist. At least “I exist” is false at worlds where the associated person moment doesn’t exist, and arguably also “I am here now”.
Yet another variation would be “I’m hungry”, “I have a headache”, “I have the visual impression of a rose”, “I’m thinking about X”. These only state something about aspects of an internal state, so their truth value only depends on the person moment, not on what the world is like apart from it. So a proposition of this sort is either true in all possible worlds where that person moment exists, or false in all possible worlds where that person moment exists (depending on whether the sensation of hungriness etc is part of the person moment or not). Though I’m not sure which truth value they should be assigned in possible worlds where the person moment doesn’t exist. If “I’m thinking of a rose” is false when I don’t exist, is “I’m not thinking of a rose” also false when I don’t exist? Both presuppose that I exist. To avoid contradictions, this would apparently require a three-valued logic, with a third truth value for propositions like that in case the associated person moment doesn’t exist.
This makes it clear that “It’s Friday” is true in some possible worlds and false in others, depending on whether my person moment (my current mental state, including all the evidence I have from perception etc) is spatio-temporally located at a Friday in that possible world.
My point is that the choice of your person moment is not part of the data of a possible world, it’s something additional to the possible world. A world contains all sorts of person moments, for many people and at many times, all together. Specifying a world doesn’t specify which of the person moments we are looking at (or from). Whether “It’s Friday” or not is a property of a (world, person-moment) pair, but not of a world considered on its own.
Keeping this distinction in mind is crucial for decision theory, since decisions are shaping the content of the world, in particular multiple agents can together shape the same world. The states of the same agent at different times or from different instances (“person moments”) can coordinate such shaping of their shared world. So the data for preference should be about what matters for determining a world, but not necessarily other things such as world-together-with-one-of-its-person-moments.
If we wanted to be super proper, then preferences should have as objects maximally specific ways the world could be, including the whole history and future of the universe, down to the last detail. Decision theory involving anything more coarse-grained than that is just a useful approximation
If X is better than Y, that gives no guidance about 40% chance of X being better or worse than 60% of Y. Preference over probability distributions holds strictly more data than preference over pure outcomes. Almost anything (such as actions-in-context) can in principle be given the semantics of a probability distribution or of an event in some space of maximally specific or partial outcomes, so preference over probability distributions more plausibly holds sufficient data.
Yeah, you’re correct—I shouldn’t have conflated “outcomes” (things utilities are non-derivatively assigned to) with “objects of preference.” Thanks for this.
As Richard Kennaway noted, it seems considerations about time are muddling things here. If we wanted to be super proper, then preferences should have as objects maximally specific ways the world could be, including the whole history and future of the universe, down to the last detail. Decision theory involving anything more coarse-grained than that is just a useful approximation—e.g. I might have a decision problem with only two outcomes being “You get $10” and “You lose $5,” but we would just be pretending these are the only two ways the world can end up for practical purposes, which is a permissible simplification since in my actual circumstances my desire to have more money over less is independent of other considerations. In particular the objects of preference are not states of the world at a time, since it is not a rational requirement that just because I prefer that one state instead of another obtain at t1 I must also prefer likewise for t2. So, our agent who is perfectly content with their eternal A-B-C cycle is just one who likes this strange history of the world they can bring about, who nevertheless has potentially acyclic preferences. Indeed, this agent is still rational on orthodox decision theory even if they’re paying for every such cycle. Similarly, in your case of the agent who goes through B to get to A, we would want to model the objects of preference as things like “Stay in the A state indefinitely” and “Be in a B-state for a little while and then be in a C-state,” and we would not want A, B, and C themselves to be the objects of preference.
So what does this say about the money-pump argument for acyclicality, which seems to treat preferences as over states-at-a-time? I think the sorts of considerations you raise do show that such arguments don’t go through unless we make certain (perhaps realistic) assumptions about the structure of agents’ preferences, assuming we view the force of the money-pump arguments to come from the possibility of actual exploitation (some proponents don’t care so much about actual exploitability, but think the hypothetical exploitability is indicative of an underlying rational inconsistency). For example, if I have the preferences A≺B≺C≺A, the argument assumes that there is some slightly worse version of A that I still prefer to C, that my preferences are coarse-grained enough that we can speak of my “trading” one of these states of affairs for another, that they are coarse-grained enough that my past trades don’t influence whether I now prefer to trade C for the slightly worse version of A, and so on. If the relevant assumptions are met, then the money-pump argument shows that my cyclic preferences leave me with an outcome that I really will disprefer to having just stuck with A, as opposed to ending up in a state I will later pay to transition out of (which is no fault of my rationality). You’re right to point out that these really are substantive assumptions.
So, in sum: I think (i) the formalism for decision theory is alright in not taking into account the possibility of transitions between outcomes, and only taking into account preferences between outcomes and which outcomes you can affect in a given decision problem; and (ii) formalism aside, your observations do show limitations to/assumptions behind the money-pump arguments insofar as those arguments are supposed to be merely pragmatic. I do not know the extent to which I disagree with you, but I hope this clarifies things.
Thank you, this is thoughtful and interesting.
As you note, you’re not the first commenter here to make these claims. But I just… really don’t buy it?
The main reason is that if you take this view, several famous and popular arguments in decision theory become complete nonsense. The two cases that come to mind immediately are money pumps for non-transitive preferences and diachronic Dutch books. In both cases, the argument assumes an agent that makes multiple successive decisions on the basis of preferences which we assume do not vary with time (or which only vary with time in some specified manner, rather than being totally unconstrained as your proposal would have it).
If sequential decisions like this were simply not the object of study in decision theory, then everyone would immediately reject these arguments by saying “you’ve misunderstood the terminology, you’re making a category error, this is just not what the theory is about.” Indeed, in this case, the arguments probably would not have ever been put forth in the first place. But in reality, these are famous and well-regarded arguments that many people take seriously.
So either there’s a clear and consistent object of study which is not these “maximally specific histories”… or decision theorists are inconsistent / confused about what the object of study is, sometimes claiming it is “maximally specific histories” and sometimes claiming otherwise.
Separately, if the theory really is about “maximally specific histories,” then the theory seems completely irrelevant to real decision-making. Outside of sci-fi or thought experiments, I will never face a “decision problem” in which both of the options are complete specifications of the way the rest of my life might go, all the way up to the moment I die.
And even if that were the sort of thing that could happen, I would find such a decision very hard to make, because I don’t find myself in possession of immediately-accessible preferences about complete life-histories; the only way I could make the decision in practice would be to “calculate” my implicit preferences about the life-histories by aggregating over my preferences about all the individual events that happen within them at particular times, i.e. the kinds of preferences that I actually use to make real-life decisions, and which I “actually have” in the sense that I immediately feel a sense of preferring one thing to another when I compare the two options.
Relatedly, it seems like very little is really “rationally required” for preferences about “maximally specific histories,” so the theory of them does not have much interesting content. We can’t make pragmatic/prudential arguments like money pumps, nor can we fall back on some set of previously held intuitions about what rational decision-making is (because “rational decision-making” in practice always occurs over time and never involves these unnatural choices between complete histories). There’s no clear reason to require the preferences to actually relate to the content of the histories in some “continuous” way; all the “physical” content of the histories, here, is bundled inside contentless labels like “A” and is thus invisible to the theory. I could have a totally nonsensical set of preferences that looks totally different at every moment in time, and yet at this level of resolution, there’s nothing wrong with it (because we can’t see the “moments” at this level, they’re bundled inside objects like “A”). So while we can picture things this way if we feel like it, there’s not much that’s worth saying about this picture of things, nor is there much that it can tell us about reality. At this point we’re basically just saying “there a set with some antisymmetric binary relation on it, and maybe we’ll have an argument about whether the relation ‘ought’ to be transitive.” There’s no philosophy left here, only a bit of dull, trivial, and unmotivated mathematics.
Thanks for the reply!
So, in my experience it’s common for decision theorists in philosophy to take preferences to be over possible worlds and probability distributions over such (the specification of which includes the past and future), and when coarse-graining they take outcomes to be sets of possible worlds. (What most philosophers do is, of course, irrelevant to the matter of how it’s best to do things, but I just want to separate “my proposal” from what I (perhaps mistakenly) take to be common.) As you say, no agent remotely close to actual agents will have preferences where details down to the location of every particle in 10,000 BC make a difference, which is why we coarse-grain. But this fact doesn’t mean that maximally-specific worlds or sets of such shouldn’t be what are modeled as outcomes, as opposed to outcomes being ways the world can be at specific times (which I take to be your understanding of things; correct me if I’m wrong). Rather, it just means a lot of these possible worlds will end up getting the same utility because the agent is indifferent between them, and so we’ll be able to treat them as the same outcome.
I worry I might be misunderstanding your point in the last few paragraphs, as I don’t understand how having maximally-specific worlds as outcomes is counter to our commonsense thinking of decision-making. It is true I don’t have a felt preference between “I get $100 and the universe was in state S five billion years ago” and “I get $100 and the universe was in state T five billion years ago,” let alone anything more specific. But that is just to say I am indifferent—I weakly prefer either outcome to the other, and this fact is as intuitively accessible to me as the fact that I prefer getting $100 to getting $10. As far as I can tell, the only consequence of making things too fine-grained is that you include irrelevant details to which the agent is indifferent and you make the model needlessly unwieldy—not that it invokes a concept of preference divorced from actual decision-making.
Every decision problem can be modeled as being between probability distributions over such histories, or over sets of such histories if we want to coarse-grain and ignore minute details to which the agent is indifferent. Now, of course, a real human never has precise probabilities associated with the most minute detail—but the same is true even if the outcomes are states-at-times, and even non-maximally-specific states-at-times. Idealizing that matter away seems to be something any model has to do, and I’m not seeing what problems are introduced specifically by taking outcomes to be histories instead of states-at-times.
But having outcomes be sets of maximally-specific world-histories doesn’t prevent us from being able to model sequential decisions. All we need to do so is to assume that the agent can make choices at different times which make a difference as to whether a more-or-less preferred world-history will obtain. For example, say I have a time-dependence preference of preferring to wear a green shirt on weekends and a red shirt on weekdays, so I’m willing to pay $1 to trade shirts every Saturday and Monday. This would not make me a money-pump, rather I’m just someone who’s willing to pay to keep their favorite shirt at the moment on. But allowing time-dependent preferences like this doesn’t prevent money-pump and diachronic Dutch book arguments from ever going through, for I still prefer wearing a red shirt next Monday to wearing a green shirt then, if everything else about the world-history is kept the same. If I in addition prefer wearing a yellow shirt on a weekday to wearing a red one, but then also prefer a green shirt on a weekday to a yellow one, then my preferences are properly cyclic and I will be a money-pump. For let’s say I have a ticket that entitles me to a red shirt next Monday; then suppose I trade that ticket for a yellow shirt ticket, then that one for a green shirt ticket, then I pay $1 to trade that for my original red shirt ticket. Assuming my preferences merely concern (i) what color shirt I wear on what day and (ii) my having more money than less, I will end up in a state that I think is worse than just having made no trade at all. This is so even though my preferences are not merely over shirt-colors and money, but rather coarse-grained sets of world-histories.
All that is needed for the pragmatic money-pump argument to go through is that my preferences are coarse-grained enough that we can speak of my making sequential choices as to which world-histories will obtain, such that facts about what choices I have made in these very cases do not influence the world-history in a way that makes a difference to my preferences. This seems pretty safe in the case of humans, and a result showing transitivity to be rationally required under such assumptions still seems important. Even in the case of a realistic agent who really likes paying to go through the A-B-C-A cycle, we could imagine alternate decision problems where they get to choose whether they get to go through that cycle during a future time-interval, and genuinely exploit them if those preferences are cyclic.
I worry that what I described above in the shirt-color example falls under what you mean by “or which only vary with time in some specified manner, rather than being totally unconstrained as your proposal would have it.” A world-history is a specification of the world-state at each time, similar in kind to “I wear a red shirt next Monday.” As I said before, the model would allow for weird fine-grained preferences over world-histories, but realistic uses of it will have as outcomes larger sets of world-histories like the set of histories where I wear a red shirt next Monday and have $1000, and so on. This is similar to how standard probability theory in principle allows one to talk about probabilities of propositions so specific that no one could ever think or care about them, but for practical purposes we just don’t include those details when we actually apply it.
I’m concerned that I may be losing track of where our disagreement is. So, just to recap: I took your arguments in the OP to be addressed if we understand outcomes (the things to which utilities are assigned) to be world-histories or sets of world-histories, so we can understand the difference between an agent who’s content with paying a dollar for every A-B-C-A cycle and an agent for whom that would be indicative of irrationality. You alleged that if we understand outcomes in this way, and not merely as possible states of the world with no reference to time, then (i) we cannot model sequential decisions, (ii) such a model would have an unrealistic portrayal of decision-making, and (iii) there will be too few constraints on preferences to get anything interesting. I reply: (i) sequential decisions can still be modeled as choices between world-histories, and indeed in an intuitive way if we make the realistic assumption that the agent’s choices in this very sequence by themselves do not make a difference to whether a world-history is preferred; (ii) the decisions do seem to be intuitively modeled if we coarse-grain the sets of world-histories appropriately; and (iii) even if money-pump arguments only go through in the case of agents who have a certain additional structure to their preferences, this additional structure is satisfied by realistic agents and so the restricted results will be important. If I am wrong about what is at issue, feel free to just correct me and ignore anything I said under false assumptions.
Preferences can be equally rigorously defined over events if probabilities and utilities are also available. Call a possible world ω, the set of all possible worlds Ω, and an a set E such that E⊆Ω an “event”. Then the utility U of E is plausibly U(E)=∑ωi∈EP(ωi∣E)U(ωi). This is a probability-weighted average, which derives from dividing the expected utility P(E)U(E)=∑ωi∈EP(ωi)U(ωi) by P(E), to arrive at the formula for U(E) alone.
So if we have both a probability function P and a utility function U over possible worlds, we can also fix a Boolean algebra of events over which those functions are defined. Then a “preference” between two events E≻F is simply U(E)>U(F).
“Events” are a lot more practical than possible worlds, since events don’t have to be maximally specific, and they correspond directly to propositions, which one can “believe” and “desire” to be true. Degrees of belief and degrees of desire can be described by probability and utility functions respectively.
Many propositions can be given event semantics, but with the caveat that events should still be parts of a space of possible worlds, so that “It’s Friday” is not an event (while “When it’s Friday, I try to check the mail” is).
Where do you think is the difference? I agree that there is a problem with indexical content, though this affects both examples. (“It’s (currently) Friday (where I live).”)
Even though it doesn’t solve all problems with indexicals, it’s probably better to not start with possible worlds but instead start with propositions directly, similar to propositional logic. Indeed this is what Richard Jeffrey does. Instead of starting with a set of possible worlds, he starts with a disjunction of two mutually exclusive propositions A and B:
U(A∨B)=P(A)U(A)+P(B)U(B)P(A)+P(B), if P(A∧B)=0,P(A∨B)≠0.
An indexical proposition doesn’t evaluate a possible world (in the sense of a maximally detailed world history), it evaluates a possible world together with a location or time or a subsystem (such as an agent/person) in it. But pointing to some location or subsystem that is already part of the world doesn’t affect the possible world itself, so it makes little sense to have preference that becomes different depending on which location or subsystem we are looking at (from outside the world), while the world remains completely the same. The events that should be objects of preference are made of worlds, not of (world, embedded subsystem) pairs.
Whether “It’s Friday” is not a property of a world, it’s a property of a world together with some admissible spacetime location. You can’t collect a set of possible world histories that centrally have the property of “It’s Friday”. On the other hand, “I try to make sure to check the mail on Fridays” is a property that does distinguish worlds where it’s centrally true (for a specific value of “I”). In general, many observations are indexical, they tell you where you are, which version of you is currently affecting the world, and a policy converts those indexical observations into an actual effect in the world that can be understood as an event, in the sense of a set of possible worlds.
I mean I agree, indexicals don’t really work with interpreting propositions simply as sets of possible worlds, but both sentences contain such indexicals, like “I”, implicitly or explicitly. “I” makes only sense for a specific person at a specific time. “It’s Friday (for me)”, relative to a person and a time, fixes a set of possible worlds where the statement is true. It’s the same for “I try to make sure to check the mail on Fridays”.
After you bundle the values of free variables into the proposition (make a closure), and “I” and such get assigned their specific referents, “It’s Friday” is still in trouble. Because if it gets the current time bundled in it, then it’s either true or false depending on the time, not depending on the world (in the maximally detailed world history sense), and so it’s either true about all worlds or none (“The worlds where it’s Friday on Tuesday”), there is no nontrivial set-of-worlds meaning. But with the mail proposition (or other propositions about policies) there is no problem like that.
I think one issue with the “person+time” context is that we may assume that once I know the time, I must know whether it is Friday or not. A more accurate assessment would be to say that an indexical proposition corresponds to a set of possible worlds together with a person moment, i.e. a complete mental state. The person moment replaces the “person + time” context. This makes it clear that “It’s Friday” is true in some possible worlds and false in others, depending on whether my person moment (my current mental state, including all the evidence I have from perception etc) is spatio-temporally located at a Friday in that possible world. This also makes intuitive sense, since I know my current mental state but that alone is not necessarily sufficient to determine the time of week, and I could be mistaken about whether it’s Friday or not.
A different case is “I am here now” or the classic “I exist”. Which would be true for any person moment and any possible world where that person moment exists. These are “synthetic a priori” propositions. Their truth can be ascertained from introspection alone (“a priori”), but they are “synthetic” rather than “analytic”, since they aren’t true in every possible world, i.e. in worlds were the person moment doesn’t exist. At least “I exist” is false at worlds where the associated person moment doesn’t exist, and arguably also “I am here now”.
Yet another variation would be “I’m hungry”, “I have a headache”, “I have the visual impression of a rose”, “I’m thinking about X”. These only state something about aspects of an internal state, so their truth value only depends on the person moment, not on what the world is like apart from it. So a proposition of this sort is either true in all possible worlds where that person moment exists, or false in all possible worlds where that person moment exists (depending on whether the sensation of hungriness etc is part of the person moment or not). Though I’m not sure which truth value they should be assigned in possible worlds where the person moment doesn’t exist. If “I’m thinking of a rose” is false when I don’t exist, is “I’m not thinking of a rose” also false when I don’t exist? Both presuppose that I exist. To avoid contradictions, this would apparently require a three-valued logic, with a third truth value for propositions like that in case the associated person moment doesn’t exist.
My point is that the choice of your person moment is not part of the data of a possible world, it’s something additional to the possible world. A world contains all sorts of person moments, for many people and at many times, all together. Specifying a world doesn’t specify which of the person moments we are looking at (or from). Whether “It’s Friday” or not is a property of a (world, person-moment) pair, but not of a world considered on its own.
Keeping this distinction in mind is crucial for decision theory, since decisions are shaping the content of the world, in particular multiple agents can together shape the same world. The states of the same agent at different times or from different instances (“person moments”) can coordinate such shaping of their shared world. So the data for preference should be about what matters for determining a world, but not necessarily other things such as world-together-with-one-of-its-person-moments.
If X is better than Y, that gives no guidance about 40% chance of X being better or worse than 60% of Y. Preference over probability distributions holds strictly more data than preference over pure outcomes. Almost anything (such as actions-in-context) can in principle be given the semantics of a probability distribution or of an event in some space of maximally specific or partial outcomes, so preference over probability distributions more plausibly holds sufficient data.
Yeah, you’re correct—I shouldn’t have conflated “outcomes” (things utilities are non-derivatively assigned to) with “objects of preference.” Thanks for this.