Thinking more about it, I think I don’t stand by my original reply. It seems possible to have some theorems whose result I currently feel 50-50 about, but which are important enough that I’m at least uncertain if I will ever be able to build a broad enough coalition of logically counter-factual beings that include people where the opposite of the theorem is true.
I think the same problem arises for some empirical questions too—T1 and T2 can be questions like “is iron’s atomic number 26 or 27?” I would have been roughly 50-50 before looking it up, but I’m uncertain if I should try to cooperate with people living in worlds where the atomic number of iron is 27 - I don’t know if those worlds are compatible with life.
However, thinking through these examples, I think I now reject the premise that updateful EDT bets wrongly in your example of the two theorems or in Paul’s original calculator example.
I think in both cases the decision-correlational reference class you should take into account is not just you learning T1 is true and you learning T2 is true within this particular experiment. It’s every instance across the multiverse where beings similar to you need to make bets about questions they have no clue about. Taking all these correlations into account, the correct thing to do is to bet with 50-50.
(As an example: when I’m betting on the atomic number of iron, I shouldn’t think of myself as cooperating with versions of myself who live in a world where iron has 27 protons. Those worlds might not exist. But I’m cooperating with instances where the game-master decided to ask if iron has 25 or 26 protons.)
Separately, at the end of the days, I still want to do acausal trade with a broad coalition of worlds which might or might not include ones where iron has 27 protons and the T1 theorem is false. But I now think that this is a separate question, and updatelessness might not be required in our mortal life.
I think the same problem arises for some empirical questions too—T1 and T2 can be questions like “is iron’s atomic number 26 or 27?” I would have been roughly 50-50 before looking it up, but I’m uncertain if I should try to cooperate with people living in worlds where the atomic number of iron is 27 - I don’t know if those worlds are compatible with life.
Minor: The question about whether those worlds are compatible with life seems like a logical rather than empirical question to me. So this still seems like an issue with logical updatelessness rather than empirical updatelessness.
As an example: when I’m betting on the atomic number of iron, I shouldn’t think of myself as cooperating with versions of myself who live in a world where iron has 27 protons. Those worlds might not exist. But I’m cooperating with instances where the game-master decided to ask if iron has 25 or 26 protons.
As in: If you’re in a counterfactual mugging where Omega says they’d reward you in a world where iron has 27 protons if you pay in this world, then you pay because you expect there to be a bunch of Omegas elsewhere doing other logical counterfactual muggings. In roughly half of those cases, someone is about to get paid by omega if their impossible counterpart pays. And your action provides evidence that their impossible counterpart pays and that Omega gives them the reward.
And the same structure applies in the calculator example and the “conjunction of two theorems” example, because you’re correlated with a bunch of other distant people where you have so little information about the details of their situation that your epistemic position is “ex-ante” relative to their dilemma, so even if you’re updateful, you bet to optimize ex-ante utility in your case, to get evidence that they bet to optimize ex-ante utility in their case.
Hm, maybe that’s right.
Doesn’t that feel really unsatisfying though? I still feel like updateful EDT recommends wrong actions in important test cases. It’s just a contingent fact that most dilemmas like this will be small-scale in a large-scale universe, and that EDT’s recommendation to act as if you double-update will be swamped by not wanting to get evidence that other people elsewhere double-updates. And there’s always going to be that force pushing towards recommending double-updating, so if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad.
I think that the situation needs to be quite extreme for my argument not to work. I think it’s quite likely I will never get to the point where I think that a decision is particularly high-stakes or universal in the grand scheme of things. I think it’s plausible that until negentropy runs out, I will always think that there is an even larger an more complicated distribution of logical counter-factual worlds out there that I haven’t explored yet, compared to which I’m only a tiny speck. So I think plausibly I will always think that I should bet 50-50 when I know nothing about something, because that’s the right policy overall.
I agree though that it’s not entirely impossible that I will come to a point where I no longer have uncertainty about what’s outside the distribution I already explored; I believe that my decision is very high stakes and doesn’t correlate with many other different decisions in my logical distribution; and I believe that worlds where T1 is false are so inconceivable that they can’t be part of my trade coalition of logically counter-factual worlds.
But I think that’s also the point where normal probabilities and betting rules entirely break down for me.
When I make a bet about a 1⁄4 probability even, I imagine it that I’m making decisions for four subagents, representing beliefs in the four different outcomes. Normally, when I bet on coinflips and other mundane questions, these four subagents love each other, and they are utilitarian about maximizing the sum of their resources. So they are okay with making bet on one outcome, which means transferring the money of three subagents to the fourth.
But if I believe that once I learn that T1 is true, I will consider in inconceivable that T1-false worlds can ever be part of my coalition, that’s a different situation. In that case, I think my T1-true and T1-false subagents don’t love each other and are indifferent to each other’s well-being. If I’m offered a bet, that’s equivalent to three subagent transferring their wealth to the fourth, and they will refuse to do that. So if I’m only offered one possible bet (betting on the conjunction of T1 and T2), I think I will bet one-fourth of my wealth on it, independently of the odds.
I agree this sounds a bit like an epicycle, but belief-representing subagents negotiating in a moral parliament is an important part of my world-view for other reasons too, (I will soon send a doc about this to you), so this solution feels quite natural to me. And it’s not like I otherwise have great intuitions about what to do at the point of meta-logical near-omniscience where I am able to tell that my current decision is high-stakes within the entire multiverse of logically counterfactual worlds.
Ok, in my eventual formulation, I’m not actually relying on moral parliaments. However, I will note that in your and Paul’s original formulation of the double-updating paradox, you were writing about bets where your utility is linear in money (e.g. small amounts of money donated to charity).
Here you write “if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad”. At the point where you are making very high-stakes and universally applicable decisions, I’m no longer convinced you get linearity, and I might bite the bullet that there might be some bets you don’t want to engage in on either side, or the paradox fails in other ways.
Altogether, I still think that my previous answer basically dissolves this paradox.
I think in both cases the decision-correlational reference class you should take into account is not just you learning T1 is true and you learning T2 is true within this particular experiment. It’s every instance across the multiverse where beings similar to you need to make bets about questions they have no clue about. Taking all these correlations into account, the correct thing to do is to bet with 50-50.
Thinking more about it, I think I don’t stand by my original reply. It seems possible to have some theorems whose result I currently feel 50-50 about, but which are important enough that I’m at least uncertain if I will ever be able to build a broad enough coalition of logically counter-factual beings that include people where the opposite of the theorem is true.
I think the same problem arises for some empirical questions too—T1 and T2 can be questions like “is iron’s atomic number 26 or 27?” I would have been roughly 50-50 before looking it up, but I’m uncertain if I should try to cooperate with people living in worlds where the atomic number of iron is 27 - I don’t know if those worlds are compatible with life.
However, thinking through these examples, I think I now reject the premise that updateful EDT bets wrongly in your example of the two theorems or in Paul’s original calculator example.
I think in both cases the decision-correlational reference class you should take into account is not just you learning T1 is true and you learning T2 is true within this particular experiment. It’s every instance across the multiverse where beings similar to you need to make bets about questions they have no clue about. Taking all these correlations into account, the correct thing to do is to bet with 50-50.
(As an example: when I’m betting on the atomic number of iron, I shouldn’t think of myself as cooperating with versions of myself who live in a world where iron has 27 protons. Those worlds might not exist. But I’m cooperating with instances where the game-master decided to ask if iron has 25 or 26 protons.)
Separately, at the end of the days, I still want to do acausal trade with a broad coalition of worlds which might or might not include ones where iron has 27 protons and the T1 theorem is false. But I now think that this is a separate question, and updatelessness might not be required in our mortal life.
Minor: The question about whether those worlds are compatible with life seems like a logical rather than empirical question to me. So this still seems like an issue with logical updatelessness rather than empirical updatelessness.
As in: If you’re in a counterfactual mugging where Omega says they’d reward you in a world where iron has 27 protons if you pay in this world, then you pay because you expect there to be a bunch of Omegas elsewhere doing other logical counterfactual muggings. In roughly half of those cases, someone is about to get paid by omega if their impossible counterpart pays. And your action provides evidence that their impossible counterpart pays and that Omega gives them the reward.
And the same structure applies in the calculator example and the “conjunction of two theorems” example, because you’re correlated with a bunch of other distant people where you have so little information about the details of their situation that your epistemic position is “ex-ante” relative to their dilemma, so even if you’re updateful, you bet to optimize ex-ante utility in your case, to get evidence that they bet to optimize ex-ante utility in their case.
Hm, maybe that’s right.
Doesn’t that feel really unsatisfying though? I still feel like updateful EDT recommends wrong actions in important test cases. It’s just a contingent fact that most dilemmas like this will be small-scale in a large-scale universe, and that EDT’s recommendation to act as if you double-update will be swamped by not wanting to get evidence that other people elsewhere double-updates. And there’s always going to be that force pushing towards recommending double-updating, so if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad.
I think that the situation needs to be quite extreme for my argument not to work. I think it’s quite likely I will never get to the point where I think that a decision is particularly high-stakes or universal in the grand scheme of things. I think it’s plausible that until negentropy runs out, I will always think that there is an even larger an more complicated distribution of logical counter-factual worlds out there that I haven’t explored yet, compared to which I’m only a tiny speck. So I think plausibly I will always think that I should bet 50-50 when I know nothing about something, because that’s the right policy overall.
I agree though that it’s not entirely impossible that I will come to a point where I no longer have uncertainty about what’s outside the distribution I already explored; I believe that my decision is very high stakes and doesn’t correlate with many other different decisions in my logical distribution; and I believe that worlds where T1 is false are so inconceivable that they can’t be part of my trade coalition of logically counter-factual worlds.
But I think that’s also the point where normal probabilities and betting rules entirely break down for me.
When I make a bet about a 1⁄4 probability even, I imagine it that I’m making decisions for four subagents, representing beliefs in the four different outcomes. Normally, when I bet on coinflips and other mundane questions, these four subagents love each other, and they are utilitarian about maximizing the sum of their resources. So they are okay with making bet on one outcome, which means transferring the money of three subagents to the fourth.
But if I believe that once I learn that T1 is true, I will consider in inconceivable that T1-false worlds can ever be part of my coalition, that’s a different situation. In that case, I think my T1-true and T1-false subagents don’t love each other and are indifferent to each other’s well-being. If I’m offered a bet, that’s equivalent to three subagent transferring their wealth to the fourth, and they will refuse to do that. So if I’m only offered one possible bet (betting on the conjunction of T1 and T2), I think I will bet one-fourth of my wealth on it, independently of the odds.
I agree this sounds a bit like an epicycle, but belief-representing subagents negotiating in a moral parliament is an important part of my world-view for other reasons too, (I will soon send a doc about this to you), so this solution feels quite natural to me. And it’s not like I otherwise have great intuitions about what to do at the point of meta-logical near-omniscience where I am able to tell that my current decision is high-stakes within the entire multiverse of logically counterfactual worlds.
Ok, in my eventual formulation, I’m not actually relying on moral parliaments. However, I will note that in your and Paul’s original formulation of the double-updating paradox, you were writing about bets where your utility is linear in money (e.g. small amounts of money donated to charity).
Here you write “if you ever get evidence that a decision is high-stakes enough and universal enough throughout the universe, EDT may well recommend that you make an exception for it and do a proper double-update on it, which seems bad”. At the point where you are making very high-stakes and universally applicable decisions, I’m no longer convinced you get linearity, and I might bite the bullet that there might be some bets you don’t want to engage in on either side, or the paradox fails in other ways.
Altogether, I still think that my previous answer basically dissolves this paradox.