Winning here corresponds to getting the most expected utility, as measured from the start of the problem. We assume we can measure utility with money, so we can e.g. put a dollar value on getting rescued.
1) In the smoking lesion problem, the exact numbers don’t so much matter as the relationship between them: you are simply choosing whether or not to gain an amount of utility or to pass it up.
2) Defecting on a cooperator in a one-shot true prisoners dilemma is the “best” outcome, so this is exactly right. See this story.
3) In the hitchhiker example, we might say that we have a negative $1,000,000 value associated with being stranded. This exact amount is usually not specified, the assumption being that this is a large negative amount.
Thus the outcomes are:
We don’t pay and are stranded in the desert, -$1,000,000
We don’t pay and the driver took us back to the city, $0
We would have paid and are stranded in the desert, -$1,000,000
We pay and the driver took us back to the city, -$100
Of course, the middle two bullet points are logically counter to the problem (assuming an open source setting), so the winning decision theories here lose only $100 as opposed to losing $1,000,000 (dying from being stranded in the desert).
Beliefs here are weakly held, I want to become more right.
I think defining winning as coming away with the most utility is a crisp measure of what makes a good decision theory.
The theory of counterfactuals are, in my mind, what separates the decision theories themselves, and is therefore the core question/fuzziness in solving decision theory. Changing your theory of counterfactuals alters the answer of the fundamental question: “when you change your action/policy, what parts of the world change with you?”.
It doesn’t seem like there is a directly objective answer to this based on the mechanic—should you change everything that’s causally downstream of you? or everything that’s logically dependent on your decision? or everything that’s correlated with your decision? A priori these all seem basically reasonable to me, until we plug these into examples and see if they are dominated by other decision theories, as measured by expected utility.
(I think?) in examples like counterfactual mugging, the measuring stick is pretty clearly whichever decision theory gets more expected utility over the whole duration of the universe. It seems fine to lose utility where you start in the middle of the scenario (operationalized by there being any sort of entanglements to outside the scenario).
In my view, the fuzziness is in finding a well-defined way to achieve the goal of lots of expected utility, not in the goal itself.
Over the course of the universe, the best decision theory is a consensus/multiple-evaluation theory. Evaluate which part of the universe you’re in, and what is the likelihood that you’re in a causally-unusaual scenario, and use the DT which gives the best outcome.
How a predictor works when your meta-DT gives different answers based on whether you’ve been predicted, I don’t know. Like a lot of adversarial(-ish) situation, the side with the most predictive power wins.
Winning here corresponds to getting the most expected utility, as measured from the start of the problem. We assume we can measure utility with money, so we can e.g. put a dollar value on getting rescued.
1) In the smoking lesion problem, the exact numbers don’t so much matter as the relationship between them: you are simply choosing whether or not to gain an amount of utility or to pass it up.
2) Defecting on a cooperator in a one-shot true prisoners dilemma is the “best” outcome, so this is exactly right. See this story.
3) In the hitchhiker example, we might say that we have a negative $1,000,000 value associated with being stranded. This exact amount is usually not specified, the assumption being that this is a large negative amount.
Thus the outcomes are:
We don’t pay and are stranded in the desert, -$1,000,000
We don’t pay and the driver took us back to the city, $0
We would have paid and are stranded in the desert, -$1,000,000
We pay and the driver took us back to the city, -$100
Of course, the middle two bullet points are logically counter to the problem (assuming an open source setting), so the winning decision theories here lose only $100 as opposed to losing $1,000,000 (dying from being stranded in the desert).
Edit: changed utility → expected utility.
I don’t think this sorts out the fundamental fuzziness of what “winning” means—what you need is a theory of counterfactuals
Beliefs here are weakly held, I want to become more right.
I think defining winning as coming away with the most utility is a crisp measure of what makes a good decision theory.
The theory of counterfactuals are, in my mind, what separates the decision theories themselves, and is therefore the core question/fuzziness in solving decision theory. Changing your theory of counterfactuals alters the answer of the fundamental question: “when you change your action/policy, what parts of the world change with you?”.
It doesn’t seem like there is a directly objective answer to this based on the mechanic—should you change everything that’s causally downstream of you? or everything that’s logically dependent on your decision? or everything that’s correlated with your decision? A priori these all seem basically reasonable to me, until we plug these into examples and see if they are dominated by other decision theories, as measured by expected utility.
(I think?) in examples like counterfactual mugging, the measuring stick is pretty clearly whichever decision theory gets more expected utility over the whole duration of the universe. It seems fine to lose utility where you start in the middle of the scenario (operationalized by there being any sort of entanglements to outside the scenario).
In my view, the fuzziness is in finding a well-defined way to achieve the goal of lots of expected utility, not in the goal itself.
Over the course of the universe, the best decision theory is a consensus/multiple-evaluation theory. Evaluate which part of the universe you’re in, and what is the likelihood that you’re in a causally-unusaual scenario, and use the DT which gives the best outcome.
How a predictor works when your meta-DT gives different answers based on whether you’ve been predicted, I don’t know. Like a lot of adversarial(-ish) situation, the side with the most predictive power wins.