The VNM independence axiom ignores the value of information
Followup to : Is risk aversion really irrational?
After reading the decision theory FAQ and re-reading The Allais Paradox I realized I still don’t accept the VNM axioms, especially the independence one, and I started thinking about what my true rejection could be. And then I realized I already somewhat explained it here, in my Is risk aversion really irrational? article, but it didn’t make it obvious in the article how it relates to VNM—it wasn’t obvious to me at that time.
Here is the core idea: information has value. Uncertainty therefore has a cost. And that cost is not linear to uncertainty.
Let’s take a first example: A is being offered a trip to Ecuador, B is being offered a great new laptop and C is being offered a trip to Iceland. My own preference is: A > B > C. I love Ecuador—it’s a fantastic country. But I prefer a laptop over a trip to Iceland, because I’m not fond of cold weather (well, actually Iceland is pretty cool too, but let’s assume for the sake of the article that A > B > C is my preference).
But now, I’m offered D = (50% chance of A, 50% chance of B) or E = (50% chance of A, 50% chance of C). The VNM independence principle says I should prefer D > E. But doing so, it forgets the cost of information/uncertainty. By choosing E, I’m sure I’ll be offered a trip—I don’t know where, but I know I’ll be offered a trip, not a laptop. By choosing D, I’m no idea on the nature of the present. I’ve much less information on my future—and that lack of information has a cost. If I know I’ll be offered a trip, I can already ask for days off at work, I can go buy a backpack, I can start doing the paperwork to get my passport. And if I know I won’t be offered a laptop, I may decide to buy one, maybe not as great as one I would have been offered, but I can still buy one. But if I chose D, I’ve much less information about my future, and I can’t optimize it as much.
The same goes for the Allais paradox: having certitude of receiving a significant amount of money ($24 000) has a value, which is present in choice 1A, but not in all others (1B, 2A, 2B).
And I don’t see why a “rational agent” should neglect the value of this information, as the VNM axioms imply. Any thought about that?
If you are an agent that exists in a timeline, then outcomes are world-histories. D is actually equal to (.5A’ + .5B’), where A’ is everything that will happen to you if you’re unsure what will happen to you for a period of time and then you go on a trip to Ecuador; and B’ is everything that will happen to you if you’re unsure what will happen to you for a period of time and then you get a laptop. Determining what A’ and B’ are requires predicting your future actions.
In the original setup, everything happens instantaneously, so there’s no period of uncertainty where you have to plan for two possible events.
Indeed, but my whole point is that when there is such a period of uncertainty (which is the case of most real world decisions, usually in a minor way, but sometimes in a very significant way), the independence principle doesn’t hold, specifically because of that period of uncertainty.
I’d rephrase it as, when there is such a period of uncertainty, you can no longer factor the problem into neat little chunks and everything gets way way more complicated to work with.
So, my point is that the independence axiom still holds. (.5A + .5B) is preferable to (.5A + .5C), where A, B, and C are world-histories where you know what’s going to happen in advance. And (.5A″ + .5C″) is preferable to (.5A’ + .5B’), where A’, B’, A″, and C″ are world-histories that involve periods of uncertainty. There is no violation of the VNM axioms.
A’, B’: You’re uncertain for a while whether you’ll go to Ecuador or whether you’ll get a laptop.
A″, C″: You’re uncertain where you will go.
A steelmanning of your position would be that a good decision-making heuristic should take into account not simply the assets that will eventually be made available to you, but the assets you’ll realistically be able to take advantage of.
So a rephrasing of your point that takes Nisan et al.’s objections into account would be: Risk-aversion is a valuable heuristic because usually uncertainty may lead to suboptimal decisions (take days off when it turns out you don’t need to, or the other way around). The Allais paradox is a case where such risk aversion isn’t justified, but some careless phrasing of the paradox (or of other situations in decision theory) the risk aversion may be justified.
The Allais paradox (both Eliezer version and the Wikipedia article) seems to not specify at all if the reward is instantaneous or delayed, so I wouldn’t say the risk aversion isn’t justified—if offered the paradox with no precision, I would say there is a chance for the reward to be instant, a chance for it to be delayed, so the risk aversion should be partially considered. It’s a bit of nitpicking, but not that much. There is no mention of time/delay in either of the paradox nor the axiom, and that seems to be a weakness to me.
And even without time, information can still have some value. If you chose 1B in the Allais paradox and lose, you can regret your choice (leading to, in fact, a <0 outcome) more than in you chose 2B and lose. Because of the information that it’s purely because of your choice you lose money. Or people can have a lower opinion of you (which may have negative consequences on your life) if they have the information too. Regretting what was a rational decision can be considered irrational, so I don’t see a problem with it being incompatible with VNM axioms. But the reaction of other people being irrational is not something you can discard the same way—if a VNM rational agent is unable to deal with human beings who aren’t perfectly rational, then there is a problem.
In short : the value of information is more important when it creates uncertainty—when the results of the lottery are delayed. But even when the results are instantaneous, information still have some value (positive or negative) that can make choosing 1A over 1B and 2B over 2A the rational choice to do in some situations.
The independence principle still holds, because (.5A’ + .5B’) is preferable to (.5A’ + .5C’).
This, and Benelliott’s similar reply, answers kilobug’s objection. But it raises a new question. As you say, outcomes are world-histories. And a history is more than just a set of events. At the very least, it’s an ordered set of events. And the order matters—the history by which one arrived at a given state matters—at least in most people’s eyes. For example, given that in outcomes X, Y, and Z, I end up with $100, I still prefer outcome X, where I earned it, to Y, where it was a gift, to Z, where I stole it. We can complicate the examples to cover any other differences that you might (wrongly) suppose explain my preference without regard to history. For example, in scenario Z let us suppose that the theft victim looks for the $100 in an old coat pocket, thinking she misplaced it, and voila! finds $100 that would otherwise never have been found. I still prefer X to Y to Z.
Given that people can rationally have preferences that make essential reference to history and to the way events came about, why can’t risk be one of those historical factors that matter? What’s so “irrational” about that?
Nothing. Whoever said there was?
If your goal is to not be a thief, then expected utility theory recommends that you do not steal.
I suspect most of us do have ‘do not steal’ preferences on the scale of a few hundred pounds or more.
On the other hand, once you get to, say, a few hundred human lives, or the fate of the entire species, then I stop caring about the journey as much. It still matters, but the amount that it matters is too small to ever have an appreciable effect on the decision. This preference may be unique to me, but if so then I weep for humanity.
A desire to avoid arriving at an outcome via thievery does not violate the Axiom of Independence. A desire to avoid arriving via a risky procedure does. However, I’m not convinced that the latter is any more irrational than the former. And I take the point of this thread to be whether obeying the Axiom really is a requirement of rationality.
So, when people say ‘risk aversion’, they can mean one of three different things:
I) I have a utility function that penalises world-histories in which I take risks.
II) I have a utility function which offers diminishing returns in some resource, so I am risk averse in that resource
III) I am risk averse in utility
Out of the three (III) is irrational and violates VNM. (II) is not irrational, and is an extremely common preference among humans wrt some things, but not others (money vs lives being the classic one). (I) is not irrational, but is pretty weird, I’m really not sure I have preferences like this, and when other people claim they do I become a bit suspicious that it is actually a case of (II) or (III).
Since we agree that (I) is not irrational, it remains to show that someone with that preference pattern (and not pattern III) still must have a VNM utility function—then my objection will be answered. Indeed, before we can even attribute “utility” to this person and thus go to case III, we must show that their preferences obey certain rules (or maybe just that their rational ones would).
I don’t think preference (I) is weird at all, though I don’t share it. Also not rare: a utility function that rewards world histories in which one takes risks. Consider that risk is either subjective or epistemic, not ontological, as used in VNM’s framework. Now consider the games involving chance that people enjoy. These either show (subjective probability interpretation of “risk”) or provide suggestive evidence toward the possibility (epistemic probability interpretation) that some people just plain like risk.
Why does it remain to be shown? How does this differ from the claim that any other preference pattern that does not violate a VNM axiom is modelled by expected utility?
Interesting. If I had to guess though, the way in which these people like risk depends on the way it is dispensed, and is probably not linear in the amount of risk.
Well if it doesn’t violate an axiom—and specifically I’m worried about Independence—then the case is proven. So let me try to explain why I think it does violate independence. The Allais Paradox provides a case where the risk undertaken depends on so-called irrelevant alternatives. Take the version from Luke’s Decision Theory FAQ. If I bet on the option (say “red or yellow”) having 34 $24000-payoff balls, whether I take a big risk depends on how many other balls are in the lottery. If there are 66 zero-payoff green balls in the urn at the same time, then I do take a risk. If there are no other balls in the urn, then I don’t. If I penalize the preferability of outcomes depending on the risk undertaken, then I will penalize the “red or yellow” bet if and only if the green balls are also involved. Say there are 33 yellow balls and 1 red one, and I get $27000 if I bet on “yellow” instead. I will penalize the outcome, bet on yellow and get $27000, in either scenario. If the penalty is not linear in the amount of risk, I could conceivably prefer to bet on yellow when the green balls are in the urn, and bet on [red or yellow] when there aren’t.
I’m not sure quite what the best response to this is, but I think I wasn’t understanding you up to this point. We seem to have a bit of a level mixing problem.
In VNM utility theory, we assign utility to outcomes, defined as a complete description of what happens, and expected utility to lotteries, defined as a probability distribution over outcomes. They are measured in the same units, but they are not the same thing and should not be compared directly.
VNM utility tells you nothing about how to calculate utility and everything about how to calculate expected utility given utility.
By my definitions of risk aversion, type (II) risk aversion is simply a statement about how you assign utility, while type (III) is an error in calculating expected utility.
Type (I), as best I understand it, seems to consist of assigning utility to a lottery. Its not so much an axiom violation as a category error, a bit like (to go back my geometry analogy) asking if two points are parallel to each other. It doesn’t violate independence, because its wrong on far too basic a level to even assess whether it violates independence.
Of course, this is made more complicated by f*ing human brains, as usual. The knowledge of having taken a risk affects our brains and may change our satisfaction with the outcome. My response to this is that it can be factored back into the utility calculation, at which point you find that getting one outcome in one lottery is not the same as getting it in another.
I may ask that you go read my conversation with kilobug elsewhere in this thread, as I think it comes down to the exact same response and I don’t feel like typing it all again.
I am indeed suggesting that an agent can assign utility, not merely expected utility, to a lottery. Note that in this sentence “utility” does not have its technical meaning(s) but simply means raw preference. With that caveat, that may be a better way of putting it than anything I’ve said so far.
You can call that a category error, but I just don’t see the mistake. Other than that it doesn’t fit the VNM theory, which would be a circular argument for its irrationality in this context.
Your point about f*ing human brains gets at my True Rejection, so thanks. And I read the conversation with kilobug. As a result I have a new idea where you may be coming from—about which I will quote Luke’s decision theory FAQ:
Emphasis added. It sounds to me like you favor a direct approach. For you, utility is not an as-if: it is a fundamentally real, interval-scale-able quality of our lives. In this scheme, the angst I feel while taking a risk is something I can assign a utility to, then shut up and (re-)calculate the expected utilities. Yes?
If you favor a direct approach, I wonder why you even care to defend the VNM axioms, or what role they play for you.
I am suggesting that this is equivalent to suggesting that two points can be parallel. It may be true for your special definition of point, but its not true for mine, and its not true for the definition the theorems refer to.
Yes, in the real world the lottery is part of the outcome, but that can be factored in with assigning utility to the outcomes, we don’t need to change our definition of utility when the existing one works (reading the rest of your post, I now see you already understand this).
I cannot see anything I have said to suggest I believe this. Interpreted descriptively, (as a statement about how people actually make decisions) I think it is utter garbage.
Interpreted prescriptively, I think I might believe it. I would at least probably say what while I like the fact that VNM axioms imply EU theory, I think I would consider EU the obviously correct way to do things even if they did not.
Yes.
Granted, if decision angst is often playing a large part in your decisions, and in particular costing you other benefits, I would strongly suggest you work on finding ways to get around this. Rightly or wrongly, yelling “stop being so irrational!” at my brain has sometimes worked here for me. I am almost certain there are better techniques.
I defend them because I think they are correct. What more reason should be required?
So let me rephrase my earlier question (poorly phrased before) about what role the VNM axioms play for you. Sometimes (especially when it comes to “rationality”) an “axiom” is held to be obvious, even indubitable: the principle of non-contradiction is often viewed in this light. At other times, say when formulating a mathematical model of an advanced physics theory, the axioms are anything but obvious, but they are endorsed because they seem to work. The axioms are the result of an inference to the best explanation.
So I’m wondering if your view is more like (A) than like (B) below.
(A) Rationality is a sort of attractor in mind-space, and people approach closer and closer to being describable by EU theory the more rational they are. Since the VNM axioms are obeyed in these cases, that tends to show that rationality includes following those axioms.
(B) Obviously only a mad person would violate the Axiom of Independence knowing full well they were doing so.
And now we are back to my True Rejection, namely: I don’t think it’s irrational to take decision-angst into account, or to seek to avoid it by avoiding risk rather than just seeking psychotherapy so that one can buck up and keep a stiff upper lip. It’s not Spock-like, but it’s not irrational.
At this point it’s important to remember that in the VNM framework, the agent’s epistemic state and decision-making procedure cannot be part of the outcome. In this sense VNM-rational agents are Cartesian dualists. Counterfactual world-histories are also not part of the outcome.
So I think whether or not a decision was risky depends on the agent’s epistemic state, as well as on the decision and the agent’s preferences. This is why preferring to come by your money honestly is different from preferring to come by your money in a non-risky way.
That’s helpful. But it also seems unduly restrictive. I realize that you’re not saying that we literally have to treat our own minds as immaterial entities (are you?), but it still seems a pretty high price to pay. Can I treat the epistemic states of my loved ones as part of the outcome? Presumably so, so why can’t I give myself the same consideration? I’m trying to make you feel the cost, here, as I see it.
Hm. I haven’t thought much about that. Maybe there is something interesting to be said about what aspects of an agent’s internal state can they have preferences over for there still to be an interesting rationality theorem? If you let agents have preferences over all decisions, then there is no rationality theorem.
I don’t believe the VNM theorem describes humans, but on the other hand I don’t think humans should endorse violations of the Independence Axiom.
Seems like a good topic to address as directly as possible, I agree.
I really really doubt you have preferences for history. Your preferences are fully summarised by the current world state with no reference to histories—you prefer not remembering having stolen something and prefer remembering having earned it and having others remember the same. Note that this is a description of the present, not of the past.
To really care about a history, you’d have to construct a scenario like “I start out in a simulation along the lines of Z, but then the simulation is rearranged to a worldstate with X instead. Alternatively, I can be in scenario X all along. I like being in state X (at a point in time after the rearrangement/lack thereof) less in the former case than in the latter, even if no one can tell the difference between them.” And I’m not sure that scenario would even work (it’s not clear that there is meaningful continuity between Z!you and X!you), but I can’t think of a better one off-hand.
Those with simpler theories of the good life often doubt the self-knowledge of those with more complex ones. There isn’t much I can do to try to convince you, other than throw thought experiments back and forth, and I don’t feel up to that. If you’ve already read EY on the complexity of value, my only thought here is that maybe some other LWers will chime in and reduce (or increase!) your posterior probability that I’m just a sloppy thinker.
In hindsight, I phrased that poorly, and you’re right, discussing it that way would probably be unproductive.
First, let me specify that when I say “histories” here I mean past histories from the point of view of the agent (which sounds weird, but a lot of the other comments use it to refer to future histories as well). With that in mind, how about this: the actions of the set of agents who care about histories are indistinguishable from the actions of some subset of the agents who do not care about histories. In (something closer to) English, there’s a way to describe your caring about histories in terms of only caring about the present and future without changing any decisions you might make.
I find the above “obvious” (which I usually take as a sign that I should be careful). The reason I believe it is that all information you have about histories is contained within your present self. There is no access to the past—everything you know about it is contained either in the present or future, so your decisions must necessarily be conditional only on the present and future.
Would you agree with that? And if so, would you agree that discussing an agent who cares about the histories leading up the present state is not worth doing, since there is no case in which her decisions would differ from some agent who does not? (I suppose one fairly reasonable objection is time travel, but I’m more interested in the case where it’s impossible, and I’m not entirely sure whether it would change the core of the argument anyway.)
That’s fair, but it just seems to show that I can be fooled. If I’m fooled and the trick is forever beyond my capacity to detect, my actions will be the same as if I had actually accomplished whatever I was trying for. But that doesn’t mean I got what I really wanted.
The problem here is that you’ve not specified the options in enough detail, for instance you appear to prefer going to Ecaudor with preparation time to going without preparation time, but you haven’t stated this anywhere. You haven’t given the slightest hint whether you prefer Iceland with preparation time to Ecuador without. VNM is not magic, if you put garbage in you get garbage out.
So to really describe the problem we need six options:
A1 - trip to Ecuador, no advance preparation A2 - trip to Ecuador, advance preparation B1 - laptop B2 - laptop, but you waste time and money preparing for a non-existant trip. C1 - trip to Iceland, no advance preparation C2 - trip to Iceland, advance preparation
Presumably you have preferences A2 > A1, B1 > B2, C2 > C1. You have also stated A > B > C, but its not clear how to interpret this, A2 > B1 > C2 seems the most charitable. You seem to also think C2 > B2, but you haven’t said so so maybe I’m wrong.
You have four possible choices, D1 = (A1 or B1), D2 = (A2 or B2), E1 = (A1 or C1) and E2 = (A2 or C2)
The VNM axioms can tell us that E2 > E1, this also seems intuitively right. If we also accept C2 > B2 then they can tell you that E2 > D2. They don’t tell us anything about how to judge between D2 and E1, since the decision here depends on the size rather than ordering of your preferences. None of this seems remotely counter-intuitive.
In short, ‘value of information’ isn’t some extra factor that needs to be taken into account on top of decision theory. It can be factored in within decision theory by correctly specifying your possible options.
Furthermore, information isn’t binary, it doesn’t suddenly appear once you have certainty and not before, if you take into account the existence of probabilistic partial information then you should find the exact same results pop out.
Why does it have value? The period where you have certainty in 1A but not in the other 3 probably only lasts a few seconds, and there aren’t any other decisions you have to make during it.
Well, sure, by mangling enough the events you can re-establish the axioms. But if you do that, in fact, you just don’t need the axioms. The independence axiom states that if you have B > C, then you have (probability p of A, probability 1-p of B) > (probability p of A, probability 1-p of C). What you’re doing is saying you can’t use A, B, and C when there is dependency, but have to create subevents like C1=”C when you are you sure you’ll have either A or C”. Of course, by splitting the events like that, you’ll reestablish independence—but by showing the need to mangle choices to make fit the axioms, you in fact have shown the axioms don’t work in the general case, when the choices you’re given are not independent, as it often is in real life.
Gwern said pretty much everything I wanted to say to this, but there’s an extra distinction I want to make
The distinction I made was things like A2=”A when you prepare” not A2=”A when you are sure of getting A or C”. This looks like a nitpick, but is in fact incredibly important. The difference between my A1 and A2, is important, they are fundamentally different outcomes which may have completely different utilities, they have no more in common than B1 and C2. They are events in their own right, there is no ‘sub-’ to it. Distinguishing between them is not ‘mangling’, putting them together in the first place was always an error.
It is easily possible to imagine three tennis players A, B and C, such that A beats B, B beats C and C beats A (perhaps A has a rather odd technique, which C has worked hard at learning to deal with despite being otherwise mediocre). Then we have A > B and B > C but not A > C, I have just shown that the axiom of transitivity is not true in the general case!
Well, no, I haven’t.
I’ve shown that the axiom of transitivity does not hold for tennis players. This may be an interesting fact about tennis, but it has not ‘disproven’ anything, nobody ever claimed that transitivity applied to tennis players.
What the VNM axioms are meant to refer to, are outcomes, meaning a complete description of what will happen to you. “Trip to Ecuador” is not an outcome, because it does not describe exactly what will happen to you, and in particular leaves open whether or not you will prepare for the trip.
This sort of thing is why I think everyone with the intellectual capacity to do so should study mathematical logic. It really helps you learn to keep things cleanly separated in your mind and avoid mistakes like this.
First, I did study mathematical logic, and please avoid such kind of ad hominem.
That said, if what you’re referring to is the whole world state, the outcomes are, in fact, awlays different. Even if only because there is somewhere in your brain the knowledge that the choice is different.
To take the formulation in the FAQ : « The independence axiom states that, for example, if an agent prefers an apple to an orange, then she must also prefer the lottery [55% chance she gets an apple, otherwise she gets cholera] over the lottery [55% chance she gets an orange, otherwise she gets cholera]. More generally, this axiom holds that a preference must hold independently of the possibility of another outcome (e.g. cholera). »
That has no meaning if you consider whole world states, not just specific outcomes. Because in the lottery it’s not “apple or orange” then but “apple with the knowledge I almost got cholera” vs “orange with the knowledge I almost got cholera”. And if there is an interaction between the two, then you have different ranking between them. Maybe you had a friend who died of cholera and loved apple, and that’ll change how much you appreciate apples knowing you almost had cholera. Maybe not. But anyway, if what you consider are whole world states, then by definition the whole world state is always different when you’re offered even a slightly different choice. How can you define an independence principle in that case ?
Fair enough
I thought this would be your reply, but didn’t want to address it because the comment was too long already.
Firstly, this is completely correct. (Well, technically we could imagine situations where the outcomes removed your memory of there ever having been a choice, but this isn’t usually the case). Its pretty much never possible to make actually useful deductions just from pure logic and the axiom of independence.
This is much the same as any other time you apply a mathematical model to the real world. We assume away some factors, not because we don’t think they exist, but because we think they do not have a large effect on the outcome or that the effect they do have does not actually affect our decision in any way.
E.g. Geometry is completely useless, because perfectly straight lines do not exist in the real world. However, in many situations they are incredibly good approximations which let us draw interesting non-trivial conclusions. This doesn’t mean Euclidean Geometry is an approximation, the approximation is when I claim the edge of my desk is a straight line.
So, I would say that usually, my memory of the other choice I was offered has quite small effects on my satisfaction with the outcome compared to what I actually get, so in most circumstances I can safely assume that the outcomes are equal (even though they aren’t). With that assumption, independence generates some interesting conclusions.
Other times, this assumption breaks down. Your cholera example strikes me as a little silly, but the example in your original post is an excellent illustration of how assuming two outcomes are equal because they look the same as English sentences can be a mistake.
At a guess, a good heuristic seems to be that after you’ve made your decision, and found out which outcome from the lottery you got, then usually the approximation that the existence of other outcomes changes nothing is correct. If there’s a long time gap between the decision and the lottery then decisions made in that time gap should usually be taken into account.
Of course, independence isn’t really that useful for its own sake, but more for the fact that combined with other axioms it gives you expected utility theory.
The cholera example was definitely a bit silly—after all, “cholera” and “apple vs orange” are usually really independent in the real world, you’ve to make very far-fetched circumstances for them to be dependent. But an axiom is supposed to be valid everywhere—even in far-fetched circumstances ;)
But overall, I understand the thing much better now: in fact, the independence principle doesn’t strictly hold in the real world, like there are no strictly right angle in the real world. But yet, like we do use the Pythagoras theorem in the real world, assuming an angle to be right when it’s “close enough” to be right, we apply the VNM axioms and the related expected utility theory when we consider the independence principle to have enough validity?
But do we have any way to measure the degree of error introduced by this approximation? Do we have ways to recognize the cases where we shouldn’t apply the expected utility theory, because we are too far from the ideal model?
My point never was to fully reject VNM and expected utility theory—I know they are useful, they work in many cases, … My point was to draw attention on a potential problem (making it an approximation, making it not always valid) that I don’t usually see being addressed (actually, I don’t remember ever having seen it that explicitly).
I think we have almost reached agreement, just a few more nitpicks I seem to have with your current post.
Its pedantic, but these two statements aren’t analogous. A better analogy would be
“the independence principle doesn’t strictly hold in the real world, like the axiom that all right angles are equal doesn’t hold in the real world”
“there are no strictly identical outcomes in the real world, like there are no strictly right angle in the real world”
Personally I prefer the second phrasing. The independence principle and the right angle principle do hold in the real world, or at least they would if the objects they talked about ever actually appeared, which they don’t.
I’m in general uncomfortable with talk of the empirical status of mathematical statements, maybe this makes me a Platonist or something. I’m much happier with talk of whether idealised mathematical objects exist in the real world, or whether things similar to them do.
What this means is we don’t apply VNM when we think independence is relatively true, we apply them when we think the outcomes we are facing are relatively similar to each other, enough that any difference can be assumed away.
This is an interesting problem. As far as I can tell, its a special case of the interesting problem of “how do we know/decide our utility function?”.
I’ve suggested one heuristic that I think is quite good. Any ideas for others?
(Once again, I want to nitpick the language. “Do we have ways to recognize the cases where two outcomes look equal but aren’t” is the correct phrasing.
‘If I write out my arithmetic like “one plus two”, your calculator can’t handle it! Proving that arithmetic doesn’t work in the general case. Sure, you can mangle these words into these things you call numbers and symbols like “1+2″, but often in real life we don’t use them.’
Hrm, could you try to steelman instead of strawmaning my position ?
It’s not just a matter of formulation or translating words to symbols. Having to split the choices offered in the real world into an undefined number of virtual choices is not just a switch of notation. Real world choices have far-fetched consequences, and having to split apart all possible interactions between those choices can easily lead to combinatorial explosion of possible choices. benelliott split my choices into 2, but he could have split them into much more : different level of preparation of the trip, buying or not a laptop by myself, …
If it means that the a VNM-based theory can’t handle directly real-life choices without having to convert them into a different set of choices, which can be much bigger than the real-life choices, well, that’s something significant that you can’t just hand-wave with ill-placed irony, and the details of the conversion process have to be part of the decision theory.
Steelman yourself. I took your quote and replaced it with an isomorphic version; it’s not my problem if it looks even more transparently irrelevant or wrong.
Yes, it is significant, but it’s along the lines of “most (optimization) problems are in complexity classes higher than P” or “AIXI is uncomputable”. It doesn’t mean that the axioms or proofs are false; it just means that, yet again, as always, we need to make trade-offs and approximations.
General rationality comment: from where I’m standing, Nisan, gwern, and benelliott have all correctly pointed out the mistake you’re making, and you don’t seem to be updating at all. Why is this? (For the record, I agree with them.)
Because in my view they did not correct any mistake I made, but they’re avoiding the core problem, using rhetoric tricks such as playing on words, irony, strawman or ad hominem instead. And I’m very disappointed to see the conversion go on this way, I wasn’t expecting that from LW. I was expecting people to disagree with me (most people here think NVM is justified) but I was expecting a constructive discussion, not such a bashing.
I don’t see Nisan or benelliott’s first comment doing any of that. (gwern could stand to be more civil.) What do you think the core problem is, and in what sense are the other comments avoiding it?
Perhaps I should elaborate on what I think the mistake is. First, let me tell you about when I made the same mistake. I once tried to point out that a rational agent may not want to do what it believes is rational if there are other agents around because it may not want to reveal to other agents information about itself. For example, if I were trying to decide between A = a trip to Ecuador and B = a trip to Iceland, I might prefer A to B but decide on B if I thought I was being watched by spies who were trying to ascertain my travel patterns and use this information against me in some way.
Someone else correctly pointed out that in this scenario I was not choosing between A and B, but between A’ = “a trip to Ecuador that spies know about” and B’ = “a trip to Iceland that spies know about,” which is a different choice. I tried to introduce a new element into the scenario without thinking about whether that element affected the outcomes I was choosing between.
I think you’re making the same kind of mistake. You start with a statement about your preferences regarding trips to Ecuador and Iceland and a new laptop, but then you introduce a new element, namely preparation time, without considering how it affects the outcomes you’re choosing between. As Nisan points out, as an agent in a timeline, you’re choosing between different world-histories, and those world-histories are more than just the instants where you get the trip or laptop. As benelliott points out, once you’ve introduced preparation time, you need to re-specify your outcomes in more detail to accommodate that, e.g. you might re-specify option E as E’ = “preparation time for a trip + 50% chance the trip is to Iceland and 50% chance the trip is to Ecuador.” And as gwern points out, any preparation you make for a trip when choosing option D will still pay off with probability 50%; you just need to consider whether that’s worth what will happen if you prepare for a trip that doesn’t happen, which is a perfectly ordinary expected value calculation.
Maybe the problem comes from my understanding of what the “alternative”, “choice” or “act” in the VNM axioms is.
To me it’s a single, atomic real-world choice you have to make: you’re offered a clear choice between options, and you’ve to select one. Like you’re offered a lottery ticket, and you can decide to buy it or not. Or to make my original example A = “in two months you’ll be given a voucher to go to Ecuador”, B = “in two months you’ll be given a laptop” and C = “in two months you’ll given a voucher to go to Iceland”. And the independence axiom that, over those choices, if I chose B over C, then I must chose (0.5A, 0.5B) over (0.5A, 0.5C). In my original understanding, things like “preparation” or “what I would do with the money if I win the lottery” are things I’m free to evaluate to chose A, B or C, but aren’t part of A, B or C.
The “world histories” view of benelliott seem to fix the problem at first glance, but to me it makes it even worse. If what you’re choosing is not individual actions, but whole “world histories”, then the independence axiom isn’t false, but doesn’t even make sense to me. Because the whole “world history” is necessarily different—the whole world history when offered to chose between B and C is in fact B’ = “B and knowing you had to chose between B and C” vs C’ = “C and knowing you had to chose between B and C”, while when offered to chose between D=(0.5A, 0.5B) vs E=(0.5A, 0.5C) is in fact (0.5A² = “A and knowing you had to chose between D and E”, 0.5B² = “B and knowing you had to chose between D and E”) vs (0.5A², 0.5C² = “C and knowing you had to chose between D and E”).
So, how do you define those (A, B, C) in the independence axiom (and the other axioms) so it doesn’t fall to the first problem, without making them factor the whole state of the world, in which case you can’t even formulate it?
To you it may be this, but the fact that this leads to an obvious absurdity suggests that this is not how most proponents think of it, or how its inventors thought of it.
I agree that things get complicated. In the worst case, you really do have to take the entire state of the world into consideration, including your own memory. For the sake of simple toy models, you can pretend that your memory is wiped after you make the choice so you don’t remember making it.
The collision I’m seeing is that between formal, mathematical axioms, and English language usage. Its clear that Benelliot is thinking of the axiom in mathematical terms: dry, inarguable, much like the independence axioms of probability: some statements about abstract sets. This is correct—the proper formulation of VNM is abstract, mathematical.
Kilobug is right in noting that information has value, ignorance has cost. But that doesn’t subvert the axiom, as the axioms are mathematically, by definition, correct; the way they were mapped to the example was incorrect: the choices aren’t truly independent.
Its also become clear that risk-aversion is essentially the same idea as “information has value”: people who are risk-averse are people who value certainty. This observation alone may well be enough to ‘explain’ the Allais paradox: the certainty of the ‘sure thing’ is worth something. All that the Allais experiment does is measure the value of certainty.
So after reading this and the comments, here is what I think: People are right that when you bundle in preparation and time delays with the uncertainty, what you’re stating is not actually an example of the paradox, because the situation is more complex than actually being a choice between 3 fungible options. On the other hand, I think you’re right if you’re pointing out that in real life you normally don’t HAVE 3 fungible options, and especially due to time and uncertainty considerations like the ones you specify, a lot of seemingly fungible options do not in fact funge. The allais paradox is an example of taking common sense thinking about the way to make YOUR kind of choices and applying it to a situation where the extra stuff that’s present in reality is not actually relevant, thus the paradox.
Does the following dialogue disprove VNM and show it’s ignoring the value of information?
“Grandma says that next Spring Break she’ll take either me or sis around the world, but to be fair she’ll flip a coin to choose!”
“That sounds pretty awesome. I hope you win. But wait, do you have a passport?”
“No. But I’ll go start the application tomorrow. It’s only like $100. I would hate to miss out on a trip around the world!”
I don’t follow this at all. You seem to be drawing some sort of bright line at ‘certainty’ vs uncertainty which seems entirely arbitrary and unjustified.
If you know that a trip is coming up which you may win but which you need to incur costs in advance, then you weigh your chance of winning and the gain from it, against the costs you incur now to make sure you can take advantage of it. If you’re offered multiple possible outcomes, you do the same thing. This is little different from, say, buying multiple kinds of insurance—except you might think of it as positive insurance.
The line isn’t arbitrary—if you’re told “you’ll receive a gift”, you’re given much less information than if you’re told “you’ll receive a trip as a gift”. The same goes here : in option D (a coin is tossed, heads you’re given a trip to Ecuador, tails you’re given a laptop) you are given much less information than in option E (a coin is tossed, heads you’re given a trip to Ecuador, tails you’re given a trip to Iceland). In option E you know you’ll be given a trip—and you can prepare for a trip. In option D, if you prepare for a trip, you’ve 50% chance of the preparation being wasted.
Now take your grandma dialogue. It needs to be added an additional option to see where it contradicts the VNM independence hypothesis: consider grandma could either offer one of us a trip around the world (T), or offer one of us a driving license (L). I value a trip around the world $2000 (in subjective dollars), and a driving license $1850 (in subjective dollars).
To make the trip around the world, as you said, I’ve to spend $100 in paperwork, but there is no such cost for driving lessons. So the total the gain $1900 if I’m offered a trip around the world, and $1850 if offered a driving lesson, just for me. I prefer T over L.
But if we are in the situation of your dialogue, I’m offered (50% chance of T) but I need to make the paperwork and spend the $100 anyway. So in fact, the total value of that offer is $2000/2-$100 = $900. While if I’m offered (50% chance of L) then the total value of that offer is $1850/2 = $925.
So I prefer T over L, but I prefer 50% chance of L to 50% chance of T. Which violates the independence principle.
Yeah, I’m going to go with benelliott on this one. You’re inputting the wrong stuff which hasn’t been split up right, and complaining that it doesn’t seem to work.
The VNM axioms are about preferences over lotteries over commodity bundles, not about preferences over lotteries over world histories. A commodity bundle is some ill-specified, immiscible utility fluid. If you want to change that model by adding an extended timeline of causal repercussions, so that there is enough room in your theory to indirectly account for instrumental preferences, then it seems to me you either have to explain what you mean by world-history bundles (UDT with its program execution histories might be thought of as having preferences over multiple world histories at once), or you should admit that you’re throwing out additivity of commodities. Additivity formalized the intuition that more goodness is better, i.e. that preferences scale with quantities of resources or intensities of emotion or frequencies of experience or whatever. That can be offloaded to the specifics of the utility function, and probably should be, but I’ve never seen anyone state that’s how they’re thinking.
There is a second, less general way some people alter VNM, by talking about preferences over lotteries over individual events within a timeline. This third form of VNM does not have space enough to account for instrumental utility unless you artificially group together causally related events, like having preferences over lotteries for “1 hour pre-travel preparation time and getting to go to Ecuador”. It’s still kind of useful for discussion because it exposes that way human preferences are similar for repeatable events, states, and experiences.