Viewed from the outside, in the logical counterfactual where the agent crosses, PA can prove its own consistency, and so is inconsistent. There is a model of PA in which “PA proves False”. Having counterfactualed away all the other models, these are the ones left. Logical counterfactualing on any statement that can’t be proved or disproved by a theory should produce the same result as adding it as an axiom. Ie logical counterfactualing ZF on choice should produce ZFC.
The only unreasonableness here comes from the agent’s worst case optimizing behaviour. This agent is excessively cautious. A logical induction agent, with PA as a deductive process will assign some prob P strictly between 0 and 1 to “PA is consistent”. Depending on which version of logical induction you run, and how much you want to cross the bridge, crossing might be worth it. (the troll is still blowing up the bridge iff PA proves False)
A logical counterfactual where you don’t cross the bridge is basically a counterfactual world where your design of logical induction assigns lower prob to “PA is consistent”. In this world it doesn’t cross and gets zero.
The alternative is a logical factual where it expects +ve util.
So if we make logical induction like crossing enough, and not mind getting blown up much, it crosses the bridge. Lets reverse this. An agent really doesn’t want blown up.
In the counterfactual world where it crosses, logical induction assigns more prob to “PA is consistant”. The expected utility procedure has to use its real probability distribution, not ask the counterfactual agent for its expected util.
I am not sure what happens after this, I think you still need to think about what you do in impossible worlds. Still working it out.
I don’t totally disagree, but see my reply to Gurkenglas as well as my reply to Andrew Sauer. Uncertainty doesn’t really save us, and the behavior isn’t really due to the worst-case-minimizing behavior. It can end up doing the same thing even if getting blown up is only slightly worse than not crossing! I’ll try to edit the post to add the argument wherein logical induction fails eventually (maybe not for a week, though). I’m much more inclined to say “Troll Bridge is too hard; we can’t demand so much of our counterfactuals” than I am to say “the counterfactual is actually perfectly reasonable” or “the problem won’t occur if we have reasonable uncertainty”.
Viewed from the outside, in the logical counterfactual where the agent crosses, PA can prove its own consistency, and so is inconsistent. There is a model of PA in which “PA proves False”. Having counterfactualed away all the other models, these are the ones left. Logical counterfactualing on any statement that can’t be proved or disproved by a theory should produce the same result as adding it as an axiom. Ie logical counterfactualing ZF on choice should produce ZFC.
The only unreasonableness here comes from the agent’s worst case optimizing behaviour. This agent is excessively cautious. A logical induction agent, with PA as a deductive process will assign some prob P strictly between 0 and 1 to “PA is consistent”. Depending on which version of logical induction you run, and how much you want to cross the bridge, crossing might be worth it. (the troll is still blowing up the bridge iff PA proves False)
A logical counterfactual where you don’t cross the bridge is basically a counterfactual world where your design of logical induction assigns lower prob to “PA is consistent”. In this world it doesn’t cross and gets zero.
The alternative is a logical factual where it expects +ve util.
So if we make logical induction like crossing enough, and not mind getting blown up much, it crosses the bridge. Lets reverse this. An agent really doesn’t want blown up.
In the counterfactual world where it crosses, logical induction assigns more prob to “PA is consistant”. The expected utility procedure has to use its real probability distribution, not ask the counterfactual agent for its expected util.
I am not sure what happens after this, I think you still need to think about what you do in impossible worlds. Still working it out.
I’ve now edited the post to address uncertainty more extensively.
I don’t totally disagree, but see my reply to Gurkenglas as well as my reply to Andrew Sauer. Uncertainty doesn’t really save us, and the behavior isn’t really due to the worst-case-minimizing behavior. It can end up doing the same thing even if getting blown up is only slightly worse than not crossing! I’ll try to edit the post to add the argument wherein logical induction fails eventually (maybe not for a week, though). I’m much more inclined to say “Troll Bridge is too hard; we can’t demand so much of our counterfactuals” than I am to say “the counterfactual is actually perfectly reasonable” or “the problem won’t occur if we have reasonable uncertainty”.