From where do those three logical nodes come from? And it looks to me like we’re not actually using the last one- am I not also entangled with agents in universes where Omega is lying about whether or not it would have provided me with $1,000, and in those cases, shouldn’t I refuse to give it $100?
On further thought, I would like to see someone explain exactly why I should give Omega $100. I’ve heard it phrased as the retroactive following-through of rational precommitments, and I’ve also heard it phrased as reflective self-consistency, in the sense that even if the coin was guaranteed to come up tails this time, we should pay because we like having Omega offer us bets where the expected value is good in general (as long as 1⁄10 coins come up heads, we break even over time, more than that and we profit). The former case, I don’t know how to handle. The latter case, I think we could represent using some form of CDT+E, or CDT over a causal-and-correlative model of beliefs.
On further thought, I would like to see someone explain exactly why I should give Omega $100.
Personally, I think all of the work is being done by Omega’s super-trustworthiness, and so I don’t think it’s a reasonable scenario to optimize for. In the real world, making a ‘rational precommitment’ on information you don’t possess seems like the reference class of ‘scams.’
(Note that I am explictly avoiding the question of what the right thing to do is; I don’t think my decision theory is currently well-equipped to handle this problem, and I’m okay with that.)
Ok, I’ve been talking it over with Benjamin Fox some more, and I don’t think Omega’s trustworthiness is the issue here. The issue is basically to come up with some decision-theoretic notion of “virtue”: “I should take action X because, timelessly speaking, a history in which I always respond to choice Y with action X nets me more money/utility/happiness than any other.” The idea is that taking action X or not doing so in any one particular instance can change which history we’re enacting, while normal decision theories reason only over the scope of a single choice-instance, with little regard for potential futures about which we don’t have specific information encoded in our causal graph.
The idea is that taking action X or not doing so in any one particular instance can change which history we’re enacting, while normal decision theories reason only over the scope of a single choice-instance, with little regard for potential futures about which we don’t have specific information encoded in our causal graph.
It seems to me that the impacts of being virtuous on one’s potential future is enough to justify being virtuous, and one does not need to take into account the impacts of being virtuous on alternative presents one might have faced instead. (Basically, instead of trusting that Omega would have given you something in an alternate world, you are trusting that human society is perceptive enough to notice and reward enough of your virtues to justify having them.)
Yes, we agree. “I will get rewarded for this behavior in the future at a profitable rate to justify my sacrifice in the present” is a reason to “self-sacrifice” in the present. The question is how to build a decision-theory that can encode this kind of knowledge without requiring actual prescience (that is, without needing to predict the specific place and time in which the agent will be rewarded).
Even using that notion of virtue, whether giving Omega the $100 benefits you only happens if Omega is trustworthy. So Omega’s trustworthiness can still be a deciding factor.
On further thought, I would like to see someone explain exactly why I should give Omega $100. I’ve heard it phrased as the retroactive following-through of rational precommitments, and I’ve also heard it phrased as reflective self-consistency, in the sense that even if the coin was guaranteed to come up tails this time, we should pay because we like having Omega offer us bets where the expected value is good in general (as long as 1⁄10 coins come up heads, we break even over time, more than that and we profit). The former case, I don’t know how to handle. The latter case, I think we could represent using some form of CDT+E, or CDT over a causal-and-correlative model of beliefs.
Personally, I think all of the work is being done by Omega’s super-trustworthiness, and so I don’t think it’s a reasonable scenario to optimize for. In the real world, making a ‘rational precommitment’ on information you don’t possess seems like the reference class of ‘scams.’
(Note that I am explictly avoiding the question of what the right thing to do is; I don’t think my decision theory is currently well-equipped to handle this problem, and I’m okay with that.)
Ok, I’ve been talking it over with Benjamin Fox some more, and I don’t think Omega’s trustworthiness is the issue here. The issue is basically to come up with some decision-theoretic notion of “virtue”: “I should take action X because, timelessly speaking, a history in which I always respond to choice Y with action X nets me more money/utility/happiness than any other.” The idea is that taking action X or not doing so in any one particular instance can change which history we’re enacting, while normal decision theories reason only over the scope of a single choice-instance, with little regard for potential futures about which we don’t have specific information encoded in our causal graph.
It seems to me that the impacts of being virtuous on one’s potential future is enough to justify being virtuous, and one does not need to take into account the impacts of being virtuous on alternative presents one might have faced instead. (Basically, instead of trusting that Omega would have given you something in an alternate world, you are trusting that human society is perceptive enough to notice and reward enough of your virtues to justify having them.)
Yes, we agree. “I will get rewarded for this behavior in the future at a profitable rate to justify my sacrifice in the present” is a reason to “self-sacrifice” in the present. The question is how to build a decision-theory that can encode this kind of knowledge without requiring actual prescience (that is, without needing to predict the specific place and time in which the agent will be rewarded).
Even using that notion of virtue, whether giving Omega the $100 benefits you only happens if Omega is trustworthy. So Omega’s trustworthiness can still be a deciding factor.
Omega’s trustworthiness mostly just means we can assign a degenerate probability of 1.0 to all information we receive from Omega.