It doesn’t seem right to list Quiggin’s rank-dependent theory and Tversky and Kahneman’s cumulative prospect theory as evidence that Independence is normatively too strong, since (IIRC) both are put forward as descriptive models of how humans actually behave, rather than normative models of how they should behave. (That said, Lara Buchak defends rank-dependent theory as a normative model (under the name ‘Risk-Weighted Expected Utility Theory.’))
3.
You don’t really reckon with the arguments against resolute choice. I like Gustafsson’s discussion in chapter 7. A summary: resolute choice either requires acting against your own preferences at the moment of choice (which seems instrumentally irrational) or else modifying your preferences (which is no defence of your original preferences).
4.
I think the Allais argument against Independence doesn’t really work. The Allais preferences can be rational if you’d feel extra disappointed getting $0 when you only had a 1% chance of doing so. But $0-with-extra-disappointment is a different outcome to $0, so those preferences don’t violate Independence!
I think the Allais argument against Independence doesn’t really work.[...] But $0-with-extra-disappointment is a different outcome to $0, so those preferences don’t violate Independence!
I strongly agree, and I think that it’s worth emphasizing that people optimize (partially) for their own emotions, and choices which seem irrational when this consideration is neglected can be rational when it is taken into account.
With that being said, there’s still a chance that an Allais-like argument could work.
Let’s imagine a different hypothetical choice:
In situation one, you choose between:
Gamble A, which is a certainty that a charity that you value will be given a million euros.
Gamble B, which is an 89% chance of one million, 10% chance of five million, 1% chance of nothing going to the same charity. In any case, it is certain that you will never find out which of these has occurred.
In situation two, you choose between:
Gamble C, which is an 11% chance of one million and an 89% chance of nothing going to the same charity, and again in any case you will never find out which of these has occurred.
Gamble D, which is a 10% chance of five million and a 90% chance of nothing going to the same charity, and again in either case you will never find out which of these has occurred.
This has a similar structure to the original Allias choice, but the 1% risk of feeling disappointment from choosing option B is gone because you’ll never find out.
If people still choose A over B and D over C here, then I think that we could conclude that people violate Independence. This is am empirical question; has a study like this ever been done?
(I leave open the question of whether this would be a mark against Independence or a mark against people’s instinctive decision-making.)
For what it’s worth, I’d still pick A over B and D over C with that change. I think I kinda compress C and D to “the charity is pretty much not gonna get any money, but on the off chance it does, might as well make it 5x more” but with A and B I still would rather they be able to work with a million rather than risk not getting anything, even if B can be compressed to “they pretty much get a lot of money, with an off chance of 5x, and a fluke chance of nothing”.
I think it might be more a question of how bad is it to not get anything? If the charity was already well funded, maybe even so funded they don’t know what to do with all the money they already have, I’d pick B and D. Likewise if I was a billionaire in the original question, I’d pick B and D. But I’m not and the charity I had in mind is not super well funded, so the cost of no money is too high when comparing A and B.
...the charity I had in mind is not super well funded...
I can see a problem with the way that I phrased the question here. I wanted an example of something that a person would value and want to make happen, but which they might plausibly not find out about. I wasn’t imagining a specific charity, but I was thinking of something linear in terms of good done per money donated, which would be something large that’s already adequately funded but not saturated. Yet one could imagine a specific charity when answering the question, and the conditions of that charity could affect the shape of the utility-vs-money curve. That means that the question could end up measuring a feature of someone’s contextual utility-vs-money curve instead of measuring their reaction to risk.
I just need an example of something that’s really good, and something else that’s five times as good, both which a person might not find out about. Maybe we could use lives saved—strangers’ lives, and you’ll never find out who—but people could have weird moral intuitions regarding saving lives that distort the results. (There’s a famous example of a framing effect, the ‘Asian disease’ problem, based on this.)
We could stick with the charity example and specify linearity in utility-vs-money, but that wouldn’t be a concise question, and it could be misunderstood.
Scenario B: 89% odds your friend gets $10, 10% odds they get dollars50 (lesswrong fucks the formatting if I use another dollar sign on this line for some reason. I’m on mobile and don’t see an option to change any formatting settings), 1% odds they get nothing
Scenario C: 11% odds your friend gets $10, 89% odds they get nothing
Scenario D: 10% odds your friend gets $50, 90% odds they get nothing
I pick B and D in these, because if my friend gets nothing in any of those scenarios it doesn’t matter. I think it really is an issue where once the guaranteed value in A gets past a certain point, almost any odds of losing it become intolerable. Maybe human values aren’t linear?
edit: and if it’s a well-funded-but-not-saturated charity, I pick B and D too, although if we’re talking about a million and 5 million it’s a tough call.
A potential reframe: certainty has a lot of value. I would not pay $10 for a plane ticket with 10% odds that I actually get to go to the place, because I can’t plan around that effectively, even if the expected value is the same as a ticket that costs dollars100 and takes me with ~100% certainty
On point 1: You’re right. The more precise statement would be: sophisticated choice avoids being exploited through a sequence of individually-accepted trades, but can still lead to ex ante dominated plans, because the agent adjusts their initial plan to accommodate foreseen future deviations rather than committing to the globally optimal plan. This is a real limitation of sophisticated choice relative to resolute choice, and I will add a note about it to the post.
On point 2: The point I’m making is about the convergence pattern rather than the original intent of any individual theory—the fact that multiple independent research programs, both descriptive and normative, all arrive at the same structural move (relax independence specifically, rather than transitivity or completeness or continuity).
On point 3:
Gustafsson’s dilemma is powerful indeed: at the moment of executing the resolute plan, either you are acting against your current preferences (which seems instrumentally irrational) or you have modified your preferences to align with the plan (in which case you’re not defending your original non-EU preferences, you’ve just adopted different ones).
I think the ergodicity economics framework provides a clean escape from both horns of this dilemma. Consider an EE agent maximizing time-average growth rate over their trajectory. At every node, their preference is the same: execute the strategy that maximizes trajectory-level growth. This preference doesn’t change at intermediate nodes, ever. The appearance of “acting against your preferences at the moment of choice” arises only if you evaluate the agent’s node-level behavior through an EU lens and ask “given that you’re at this node, doesn’t a different action have higher conditional expected utility?” But the agent’s actual preference was never about conditional expected utility at individual nodes. Their preference is about the trajectory as a whole, and that preference is entirely stable throughout the process.
So the EE agent is neither acting against their current preferences (horn a), since their trajectory-level preference consistently favors the same action at every node, nor modifying their preferences (horn b), since their preference was always trajectory-level and never changed. The dilemma’s force, how I see it, depends on assuming that the “real” preferences at any node must be the ones that EU would assign conditional on being at that node. Rejecting that assumption, which is precisely what rejecting the independence axiom amounts to, dissolves the dilemma.
On point 4:
I think we discussed it already in other comments. But in a nutshell, what I mean is that this is a well-known defense, and it has a well-known cost: it makes the independence axiom unfalsifiable. If any apparent violation can be resolved by saying “the outcomes are actually different because of the context-dependent psychological state” (disappointment, regret, elation from near-misses), then no possible pattern of behavior could ever count as a real independence violation. Any behavior whatsoever can be accommodated by enriching the outcome space with the right context-dependent psychological states.
This is fine if we want independence to be a definitional truth, but then it carries no normative force: it cannot tell us that any particular pattern of behavior is irrational, because every pattern can be rationalized by the appropriate outcome redefinition. And it cannot do any predictive or explanatory work, because it accommodates everything and therefore constrains nothing.
This is the motte-and-bailey structure I discuss in the article when talking about Academian’s and Fallenstein’s posts.
Overall: In its most general form (where outcomes can encode arbitrary contextual and psychological information), EUT is unfalsifiable.
In its actual applied form, it is falsified by the Allais pattern.
Really nice post. A few things though:
1.
This isn’t right, at least on the usual definition of ‘money pump’ where an agent is money-pumped if and only if they “end up paying for something they could have kept for free even though they knew in advance what decision problem they were facing.” As you say, sophisticated choosers who violate Independence sometimes have to settle for plans that are dominated from the ex ante perspective. That’s a money pump on the usual definition.
2.
It doesn’t seem right to list Quiggin’s rank-dependent theory and Tversky and Kahneman’s cumulative prospect theory as evidence that Independence is normatively too strong, since (IIRC) both are put forward as descriptive models of how humans actually behave, rather than normative models of how they should behave. (That said, Lara Buchak defends rank-dependent theory as a normative model (under the name ‘Risk-Weighted Expected Utility Theory.’))
3.
You don’t really reckon with the arguments against resolute choice. I like Gustafsson’s discussion in chapter 7. A summary: resolute choice either requires acting against your own preferences at the moment of choice (which seems instrumentally irrational) or else modifying your preferences (which is no defence of your original preferences).
4.
I think the Allais argument against Independence doesn’t really work. The Allais preferences can be rational if you’d feel extra disappointed getting $0 when you only had a 1% chance of doing so. But $0-with-extra-disappointment is a different outcome to $0, so those preferences don’t violate Independence!
I strongly agree, and I think that it’s worth emphasizing that people optimize (partially) for their own emotions, and choices which seem irrational when this consideration is neglected can be rational when it is taken into account.
With that being said, there’s still a chance that an Allais-like argument could work.
Let’s imagine a different hypothetical choice:
This has a similar structure to the original Allias choice, but the 1% risk of feeling disappointment from choosing option B is gone because you’ll never find out.
If people still choose A over B and D over C here, then I think that we could conclude that people violate Independence. This is am empirical question; has a study like this ever been done?
(I leave open the question of whether this would be a mark against Independence or a mark against people’s instinctive decision-making.)
That’s a cool idea! I’m not aware of any study like that, but I’d be very interested to see the results.
For what it’s worth, I’d still pick A over B and D over C with that change. I think I kinda compress C and D to “the charity is pretty much not gonna get any money, but on the off chance it does, might as well make it 5x more” but with A and B I still would rather they be able to work with a million rather than risk not getting anything, even if B can be compressed to “they pretty much get a lot of money, with an off chance of 5x, and a fluke chance of nothing”.
I think it might be more a question of how bad is it to not get anything? If the charity was already well funded, maybe even so funded they don’t know what to do with all the money they already have, I’d pick B and D. Likewise if I was a billionaire in the original question, I’d pick B and D. But I’m not and the charity I had in mind is not super well funded, so the cost of no money is too high when comparing A and B.
I can see a problem with the way that I phrased the question here. I wanted an example of something that a person would value and want to make happen, but which they might plausibly not find out about. I wasn’t imagining a specific charity, but I was thinking of something linear in terms of good done per money donated, which would be something large that’s already adequately funded but not saturated. Yet one could imagine a specific charity when answering the question, and the conditions of that charity could affect the shape of the utility-vs-money curve. That means that the question could end up measuring a feature of someone’s contextual utility-vs-money curve instead of measuring their reaction to risk.
I just need an example of something that’s really good, and something else that’s five times as good, both which a person might not find out about. Maybe we could use lives saved—strangers’ lives, and you’ll never find out who—but people could have weird moral intuitions regarding saving lives that distort the results. (There’s a famous example of a framing effect, the ‘Asian disease’ problem, based on this.)
We could stick with the charity example and specify linearity in utility-vs-money, but that wouldn’t be a concise question, and it could be misunderstood.
Does anyone have any better ideas?
Scenario A: your friend gets $10
Scenario B: 89% odds your friend gets $10, 10% odds they get dollars50 (lesswrong fucks the formatting if I use another dollar sign on this line for some reason. I’m on mobile and don’t see an option to change any formatting settings), 1% odds they get nothing
Scenario C: 11% odds your friend gets $10, 89% odds they get nothing
Scenario D: 10% odds your friend gets $50, 90% odds they get nothing
I pick B and D in these, because if my friend gets nothing in any of those scenarios it doesn’t matter. I think it really is an issue where once the guaranteed value in A gets past a certain point, almost any odds of losing it become intolerable. Maybe human values aren’t linear?
edit: and if it’s a well-funded-but-not-saturated charity, I pick B and D too, although if we’re talking about a million and 5 million it’s a tough call.
A potential reframe: certainty has a lot of value. I would not pay $10 for a plane ticket with 10% odds that I actually get to go to the place, because I can’t plan around that effectively, even if the expected value is the same as a ticket that costs dollars100 and takes me with ~100% certainty
Thank you for the careful engagement!
On point 1: You’re right. The more precise statement would be: sophisticated choice avoids being exploited through a sequence of individually-accepted trades, but can still lead to ex ante dominated plans, because the agent adjusts their initial plan to accommodate foreseen future deviations rather than committing to the globally optimal plan. This is a real limitation of sophisticated choice relative to resolute choice, and I will add a note about it to the post.
On point 2: The point I’m making is about the convergence pattern rather than the original intent of any individual theory—the fact that multiple independent research programs, both descriptive and normative, all arrive at the same structural move (relax independence specifically, rather than transitivity or completeness or continuity).
On point 3:
Gustafsson’s dilemma is powerful indeed: at the moment of executing the resolute plan, either you are acting against your current preferences (which seems instrumentally irrational) or you have modified your preferences to align with the plan (in which case you’re not defending your original non-EU preferences, you’ve just adopted different ones).
I think the ergodicity economics framework provides a clean escape from both horns of this dilemma. Consider an EE agent maximizing time-average growth rate over their trajectory. At every node, their preference is the same: execute the strategy that maximizes trajectory-level growth. This preference doesn’t change at intermediate nodes, ever. The appearance of “acting against your preferences at the moment of choice” arises only if you evaluate the agent’s node-level behavior through an EU lens and ask “given that you’re at this node, doesn’t a different action have higher conditional expected utility?” But the agent’s actual preference was never about conditional expected utility at individual nodes. Their preference is about the trajectory as a whole, and that preference is entirely stable throughout the process.
So the EE agent is neither acting against their current preferences (horn a), since their trajectory-level preference consistently favors the same action at every node, nor modifying their preferences (horn b), since their preference was always trajectory-level and never changed. The dilemma’s force, how I see it, depends on assuming that the “real” preferences at any node must be the ones that EU would assign conditional on being at that node. Rejecting that assumption, which is precisely what rejecting the independence axiom amounts to, dissolves the dilemma.
On point 4:
I think we discussed it already in other comments. But in a nutshell, what I mean is that this is a well-known defense, and it has a well-known cost: it makes the independence axiom unfalsifiable. If any apparent violation can be resolved by saying “the outcomes are actually different because of the context-dependent psychological state” (disappointment, regret, elation from near-misses), then no possible pattern of behavior could ever count as a real independence violation. Any behavior whatsoever can be accommodated by enriching the outcome space with the right context-dependent psychological states.
This is fine if we want independence to be a definitional truth, but then it carries no normative force: it cannot tell us that any particular pattern of behavior is irrational, because every pattern can be rationalized by the appropriate outcome redefinition. And it cannot do any predictive or explanatory work, because it accommodates everything and therefore constrains nothing.
This is the motte-and-bailey structure I discuss in the article when talking about Academian’s and Fallenstein’s posts.
Overall: In its most general form (where outcomes can encode arbitrary contextual and psychological information), EUT is unfalsifiable.
In its actual applied form, it is falsified by the Allais pattern.