I don’t have an elegant fix for this, but I came up with a kludgy decision procedure that would not have the issue.
Problem: you don’t want to give up a decent chance of something good, for something even better that’s really unlikely to happen, no matter how much better that thing is.
Solution: when evaluating the utility of a probabilistic combination of outcomes, instead of taking the average of all of them, remove the top 5% (this is a somewhat arbitrary choice) and find the average utility of the remaining outcomes.
For example, assume utility is proportional to lifespan. If offered a choice between a 1% chance of a million years (otherwise death in 20 years) and certainty of a 50 year lifespan, choose the latter, since the former, once the top 5% is removed, has the utility of a 20 year lifespan. If offered a choice between a 10% chance of a 19,000 year lifespan (otherwise immediate death) and a certainty of a 500 year lifespan, choose the former, since once the top 5% is removed, it is equivalent in utility to (5/95)*19,000 years, or 1,000 years.
New problem: two decisions, each correct by the decision rule, can combine into an incorrect decision by the decision rule.
For example, assume utility is proportional to money, and you start with $100. If you’re offered a 10% chance of multiplying your money by 100, otherwise losing it, and then as a separate decision offered the same deal based on your new amount of money, you’d take the offer both times, ending up with a 1% chance of having a million dollars, whereas if directly offered the 1% chance of a million dollars, otherwise losing your $100, you wouldn’t take it.
Solution: you choose your entire combination of decisions to obey the rule, not individual decisions. As with updateless decision theory, the decisions for different circumstances are chosen to maximise a function weighted over all possible circumstances (maybe starting when you adopt the decision procedure) not just over the circumstance you find yourself in.
In the example above, you could decide to take the first offer but not the second, or if you had a random number generator, take the first offer and then maybe take the second with a certain probability.
A similar procedure, choosing a cutoff probability for low utility events, can solve Pascal’s Wager. Ignoring the worst 5% of events seems too much, it may be better to pick a smaller number, though there’s no objective justification for what to pick.
I don’t have an elegant fix for this, but I came up with a kludgy decision procedure that would not have the issue.
Problem: you don’t want to give up a decent chance of something good, for something even better that’s really unlikely to happen, no matter how much better that thing is.
Solution: when evaluating the utility of a probabilistic combination of outcomes, instead of taking the average of all of them, remove the top 5% (this is a somewhat arbitrary choice) and find the average utility of the remaining outcomes.
For example, assume utility is proportional to lifespan. If offered a choice between a 1% chance of a million years (otherwise death in 20 years) and certainty of a 50 year lifespan, choose the latter, since the former, once the top 5% is removed, has the utility of a 20 year lifespan. If offered a choice between a 10% chance of a 19,000 year lifespan (otherwise immediate death) and a certainty of a 500 year lifespan, choose the former, since once the top 5% is removed, it is equivalent in utility to (5/95)*19,000 years, or 1,000 years.
New problem: two decisions, each correct by the decision rule, can combine into an incorrect decision by the decision rule.
For example, assume utility is proportional to money, and you start with $100. If you’re offered a 10% chance of multiplying your money by 100, otherwise losing it, and then as a separate decision offered the same deal based on your new amount of money, you’d take the offer both times, ending up with a 1% chance of having a million dollars, whereas if directly offered the 1% chance of a million dollars, otherwise losing your $100, you wouldn’t take it.
Solution: you choose your entire combination of decisions to obey the rule, not individual decisions. As with updateless decision theory, the decisions for different circumstances are chosen to maximise a function weighted over all possible circumstances (maybe starting when you adopt the decision procedure) not just over the circumstance you find yourself in.
In the example above, you could decide to take the first offer but not the second, or if you had a random number generator, take the first offer and then maybe take the second with a certain probability.
A similar procedure, choosing a cutoff probability for low utility events, can solve Pascal’s Wager. Ignoring the worst 5% of events seems too much, it may be better to pick a smaller number, though there’s no objective justification for what to pick.