I think that the situation needs to be quite extreme for my argument not to work. I think it’s quite likely I will never get to the point where I think that a decision is particularly high-stakes or universal in the grand scheme of things. I think it’s plausible that until negentropy runs out, I will always think that there is an even larger an more complicated distribution of logical counter-factual worlds out there that I haven’t explored yet, compared to which I’m only a tiny speck. So I think plausibly I will always think that I should bet 50-50 when I know nothing about something, because that’s the right policy overall.
I agree though that it’s not entirely impossible that I will come to a point where I no longer have uncertainty about what’s outside the distribution I already explored; I believe that my decision is very high stakes and doesn’t correlate with many other different decisions in my logical distribution; and I believe that worlds where T1 is false are so inconceivable that they can’t be part of my trade coalition of logically counter-factual worlds.
But I think that’s also the point where normal probabilities and betting rules entirely break down for me.
When I make a bet about a 1⁄4 probability even, I imagine it that I’m making decisions for four subagents, representing beliefs in the four different outcomes. Normally, when I bet on coinflips and other mundane questions, these four subagents love each other, and they are utilitarian about maximizing the sum of their resources. So they are okay with making bet on one outcome, which means transferring the money of three subagents to the fourth.
But if I believe that once I learn that T1 is true, I will consider in inconceivable that T1-false worlds can ever be part of my coalition, that’s a different situation. In that case, I think my T1-true and T1-false subagents don’t love each other and are indifferent to each other’s well-being. If I’m offered a bet, that’s equivalent to three subagent transferring their wealth to the fourth, and they will refuse to do that. So if I’m only offered one possible bet (betting on the conjunction of T1 and T2), I think I will bet one-fourth of my wealth on it, independently of the odds.
I agree this sounds a bit like an epicycle, but belief-representing subagents negotiating in a moral parliament is an important part of my world-view for other reasons too, (I will soon send a doc about this to you), so this solution feels quite natural to me. And it’s not like I otherwise have great intuitions about what to do at the point of meta-logical near-omniscience where I am able to tell that my current decision is high-stakes within the entire multiverse of logically counterfactual worlds.
Ok, in my eventual formulation, I’m not actually relying on moral parliaments. However, I will note that in your and Paul’s original formulation of the double-updating paradox, you were writing about bets where your utility is linear in money (e.g. small amounts of money donated to charity).
Here you write “if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad”. At the point where you are making very high-stakes and universally applicable decisions, I’m no longer convinced you get linearity, and I might bite the bullet that there might be some bets you don’t want to engage in on either side, or the paradox fails in other ways.
Altogether, I still think that my previous answer basically dissolves this paradox.
I think in both cases the decision-correlational reference class you should take into account is not just you learning T1 is true and you learning T2 is true within this particular experiment. It’s every instance across the multiverse where beings similar to you need to make bets about questions they have no clue about. Taking all these correlations into account, the correct thing to do is to bet with 50-50.
I think that the situation needs to be quite extreme for my argument not to work. I think it’s quite likely I will never get to the point where I think that a decision is particularly high-stakes or universal in the grand scheme of things. I think it’s plausible that until negentropy runs out, I will always think that there is an even larger an more complicated distribution of logical counter-factual worlds out there that I haven’t explored yet, compared to which I’m only a tiny speck. So I think plausibly I will always think that I should bet 50-50 when I know nothing about something, because that’s the right policy overall.
I agree though that it’s not entirely impossible that I will come to a point where I no longer have uncertainty about what’s outside the distribution I already explored; I believe that my decision is very high stakes and doesn’t correlate with many other different decisions in my logical distribution; and I believe that worlds where T1 is false are so inconceivable that they can’t be part of my trade coalition of logically counter-factual worlds.
But I think that’s also the point where normal probabilities and betting rules entirely break down for me.
When I make a bet about a 1⁄4 probability even, I imagine it that I’m making decisions for four subagents, representing beliefs in the four different outcomes. Normally, when I bet on coinflips and other mundane questions, these four subagents love each other, and they are utilitarian about maximizing the sum of their resources. So they are okay with making bet on one outcome, which means transferring the money of three subagents to the fourth.
But if I believe that once I learn that T1 is true, I will consider in inconceivable that T1-false worlds can ever be part of my coalition, that’s a different situation. In that case, I think my T1-true and T1-false subagents don’t love each other and are indifferent to each other’s well-being. If I’m offered a bet, that’s equivalent to three subagent transferring their wealth to the fourth, and they will refuse to do that. So if I’m only offered one possible bet (betting on the conjunction of T1 and T2), I think I will bet one-fourth of my wealth on it, independently of the odds.
I agree this sounds a bit like an epicycle, but belief-representing subagents negotiating in a moral parliament is an important part of my world-view for other reasons too, (I will soon send a doc about this to you), so this solution feels quite natural to me. And it’s not like I otherwise have great intuitions about what to do at the point of meta-logical near-omniscience where I am able to tell that my current decision is high-stakes within the entire multiverse of logically counterfactual worlds.
Ok, in my eventual formulation, I’m not actually relying on moral parliaments. However, I will note that in your and Paul’s original formulation of the double-updating paradox, you were writing about bets where your utility is linear in money (e.g. small amounts of money donated to charity).
Here you write “if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad”. At the point where you are making very high-stakes and universally applicable decisions, I’m no longer convinced you get linearity, and I might bite the bullet that there might be some bets you don’t want to engage in on either side, or the paradox fails in other ways.
Altogether, I still think that my previous answer basically dissolves this paradox.