The core of the 5-and-10 problem is not specific to a particular formalization or agent algorithm. It’s fundametally the question of what’s going on with agent’s reasoning inside the 5 world. In the 10 world, agent’s reasoning proceeds in a standard way, perhaps the agent considers both the 5 and 10 worlds, evaluates them, and decides to go with 10. But what might the agent be thinking in the 5 world, so that it ends up making that decision? And if the agent in the 10 world is considering the 5 world, what does the agent in the 10 world think about the thinking of the agent in the 5 world, and about what that implies in general?
How this happens is a test for decision making algorithms, as it might lead to a breakdown along the lines of the 5-and-10 problem, or to a breakdown of an informal model of how a particular algorithm works. The breakdown is not at all inevitable, and usually the test can’t even be performed without changing the algorithm to make it possible, in which case we’ve intentionally broken the algorithm in an interesting way that might tell us something instructive.
In the post, what agent algorithm are you testing? Note that agent’s actions are not the same thing as agent’s knowledge of them. Proving A = 5 in a possibly inconsistent system is not the same thing as actually doing 5 (perhaps the algorithm explicitly says to do 10 upon proving A = 5, which is the chicken rule; there is no relevant typo in this parenthetical).
Yeah sure, like there’s a logical counterfactual strand of the argument but that’s not the topic I’m really addressing here—I find those a lot less convincing so my issue here is around the use of Lobian uncertainty specifically. There’s an step very specific to this species of argument that proving that □P will make P true when P is about the outcomes of the bets, because you will act based on the proof of P.
This is invoking Lob’s Theorem in a manner which is very different from the standard counterpossible principle of explosion stuff. And I’m really wanting to discuss that step specifically because I don’t think it’s valid, and if the above argument is still representative of at least a strand of relevant argument then I’d be grateful for some clarification on how (3.) is supposed to be provable by the agent, or how my subsequent points are invalid.
The core of the 5-and-10 problem is not specific to a particular formalization or agent algorithm. It’s fundametally the question of what’s going on with agent’s reasoning inside the 5 world. In the 10 world, agent’s reasoning proceeds in a standard way, perhaps the agent considers both the 5 and 10 worlds, evaluates them, and decides to go with 10. But what might the agent be thinking in the 5 world, so that it ends up making that decision? And if the agent in the 10 world is considering the 5 world, what does the agent in the 10 world think about the thinking of the agent in the 5 world, and about what that implies in general?
How this happens is a test for decision making algorithms, as it might lead to a breakdown along the lines of the 5-and-10 problem, or to a breakdown of an informal model of how a particular algorithm works. The breakdown is not at all inevitable, and usually the test can’t even be performed without changing the algorithm to make it possible, in which case we’ve intentionally broken the algorithm in an interesting way that might tell us something instructive.
In the post, what agent algorithm are you testing? Note that agent’s actions are not the same thing as agent’s knowledge of them. Proving A = 5 in a possibly inconsistent system is not the same thing as actually doing 5 (perhaps the algorithm explicitly says to do 10 upon proving A = 5, which is the chicken rule; there is no relevant typo in this parenthetical).
Yeah sure, like there’s a logical counterfactual strand of the argument but that’s not the topic I’m really addressing here—I find those a lot less convincing so my issue here is around the use of Lobian uncertainty specifically. There’s an step very specific to this species of argument that proving that □P will make P true when P is about the outcomes of the bets, because you will act based on the proof of P.
This is invoking Lob’s Theorem in a manner which is very different from the standard counterpossible principle of explosion stuff. And I’m really wanting to discuss that step specifically because I don’t think it’s valid, and if the above argument is still representative of at least a strand of relevant argument then I’d be grateful for some clarification on how (3.) is supposed to be provable by the agent, or how my subsequent points are invalid.