Of course you’d care about the average over all the circumstances you might be in—that’s what maximizing expected utility is.
I don’t understand what your statement means in the presence of indexical uncertainty, and anyway it doesn’t seem to be a correct summary of Stuart’s idea. He only takes the average within each branch where all kids get the same outcome anyway, not the average between branches. The averaging here is justified, not by referring to indexical probabilities (which we don’t understand), but by noting that it preserves the utility obtained by each agent in each possible world. After that Stuart uses UDT (not another layer of averaging, which would be unjustified) to obtain the “correct” decision in the aggregate for all branches. Reread his solution to model 4 and then model 1 which it refers to—the reasoning is quite tricky.
Ah, sorry, I’m not very familiar with UDT. Avoiding indexical probabilities (they aren’t that poorly understood) seems to be at the root of the unintuitive answer you cite, though: would you still give your world being world 1 a probability of 1⁄2 if there were 10000 kids in world 2?
What use is intuition to anyone? Since you mentioned your intuitive reaction to the UDT answer in your post, your intuition seems to have at least a little credibility.
I could talk about the logical reasons to assign greater chance of being in circumstances with more people, if you’d prefer.
I think that would be counterproductive too, unless you have already reviewed the previous discussions of anthropics on LW and are certain that your argument is new.
We don’t understand much about indexicals right now, but we do understand that questions about subjective probabilities should be reformulated as decision problems, otherwise you get confused on easy problems like Sleeping Beauty and never get to the challenging stuff. The reason: we all agree now that decision problems ask which precommitment would be viewed by the agent as most beneficial (Eliezer’s intuition behind TDT), and precommitments take place before you look at any evidence, indexical or otherwise.
Do you feel that talking about probabilities instead of decisions brings new clarity to our problem? Then explain.
ETA: maybe rwallace’s post The I-Less Eye will be helpful to you.
Well, both the numbers and the logical steps required are identical in this case, no matter whether you think about it as decision theory or probability. The reason Sleeping Beauty is tricky is that assigning probabilities the obvious way leads to breaking normalization, but that’s not a problem in simple cases like this.
Either way, you should still use the fact that you have an equal chance of being any person whose circumstances fit yours, lacking any other information (this can be shown to be the correct prior, following from the principle of indifference, which says that if the information in a case is identical, the probability should be too).
Correct as in uniquely fulfills the desiderata of probability theory, on which the whole thing can be based. Ooh, found a link, I didn’t know that was online. Particularly important for these purposes is the principle that says that states with identical information should be assigned identical probabilities. You just know that you are one of the people in the problem, which breaks the symmetry between the two coin-flip outcomes (since there are different numbers of people depending on the outcome), but it creates a symmetry between all the states specified by “you are one of the people in this problem.”
It’s not that I’m saying it’s wrong to approach this as a decision problem. Just that ordinary probability applies fine in this case. If you get a different result with a decision theory than with bayesian probability, though, that is bad. Bad in the sense of provably worse by the measure of expected utility, unless circumstances are very extreme.
What’s “expected utility” in situations with indexical uncertainty? If you take the “expectation” according to an equal weighting of all indistinguishable observer-moments, isn’t your reasoning circular?
Also I’m interested in hearing your response to rwallace’s scenario, which seems to show that assigning equal probabilities to indistinguishable observer-moments leads to time-inconsistency.
Ah, wait: the probability in rwallace’s post was 1⁄99 all along. This is because each time you do the “clone me” operation, it’s not like it automatically halves the probability; it only works that way for a very certain set of evidence. When you are careful and specify which evidence is available, the apparent problem is resolved.
Hmm, actually I might be making the wrong correction since this would contradict the rule that P(AB) = P(A)*P(B|A). But my plan was to specify the “anthropic evidence” (memories, body, etc.) as being exactly the same stuff that makes you “you” at the start of the process, then clarifying the question as P(original | anthropic evidence).
Upon reflection, this is very shaky, but still possibly correct. I’ll try and formalize the change to the product rule and see what it says about Sleeping Beauty, which has a relatively known answer.
Oh, sorry; the calculations are trivial. It’s the parts that aren’t math that are the problem.
Take a person imagining whether to get copied 98 times. He wants to know “after I get out of the machine but before I get told if I’m the original, what is the probability that I am the orginial?” There are two different “edge cases.”
1) If during the copying process you generate no new evidence, i.e. all your copies get near-identical memories- if that’s the case, then the principle of indifference applies with overwhelming obviousness. You have nothing with which to differentiate yourself, so you must assign an equal probability, so the probability that you’re the original is 1⁄99.
2) You have to go in in 98 different sessions to get copied, thus generating extra evidence (memories) in between each time. Here, the only place you can apply the principle of indifference is within each session, so you have a probability of 1⁄2 of being the original after every session. The product rule then says that since P(AB) = P(A)*P(B|A), your final probability of being the original is (1/2)^98. But this feels odd because at no time during the process could this probability be realized—when waking up you always have a probability estimate of 1⁄2 - rendering this immune from the sort of betting game you might play in e.g. the Sleeping Beauty problem.
With these two cases in mind, we can consider a copying scheme with the causal structure of (2) but the evidence of (1) (copied in series, but no distinguishing memories). Logic would say that the evidence wins, since the evidence usually wins. But either the product rule stops working or the probability P(B|A) changes in an unusual way that must exactly parallel what the evidence requires—this provides a license for just using the evidence and not using the product rule in these sorts of cases, but it would be interesting to see whether it’s possible to save the product rule.
Saving the product rule would require changing the P(B|A) in P(AB)=P(A)*P(B|A). If we just look at the case where you’re copied 2 times in a row with identical memories, A = original the first time and B = original the second time. One woudl expect that P(A) = 1⁄2. But P(AB)=1/3, so P(B|A) must equal 3⁄4. For 98 copies the last conditional probability would be 97/98! This is really weird. So I’m still thinking about it.
Hey, I just had this idea. To get the required values of P(B|A) you may try to consider the possibility that the “subjective continuation” of a copy (which was made from the original) can jump into another copy (also made from the original, not from the first copy). There seems to be no apriori reason why that shouldn’t happen, if the information states of the copies are equivalent. Why focus on the physical continuity within the copying process anyway? Information is all that matters.
This way you get to keep the illusion that you “are” one specific copy at all times, rather the set of all identical information states at once (my preferred point of view up till now). I wonder if some other thought experiment could break that illusion more conclusively.
Hm, I guess it is circular. Dang. The question is really “is bayesian probability correct in general?”
Do you mean rwallace’s scenario with the copies? The probabilities seem correct, though since there are multiple copies, normalization might be broken (or artificially enforced) somewhere I didn’t notice, like in Sleeping Beauty. I’m a bit unsure. What is clear is that there isn’t actually a discontinuity at short times—since the probability comes form the evidence of your memories, not how it “really happened.”
EDIT: There does appear to be an inconsistency when making serial copies—computing it different ways gives different answers. Freaking normalization.
if the information in a case is identical, the probability should be too
So if I have a biased coin that comes up heads 90% of the time, flip it and hide the result from you, your credence for heads should be 50%? Nah. I’m willing to believe in the principle of indifference only if the cases are completely identical, not just observably identical. In our problem the cases aren’t completely identical—you may be the sole child or one of 10 children, and these correspond to very different and non-symmetrical states of the world.
It seems to me that you’re confused. Sleeping Beauty isn’t tricky at all, but our problem is. If rwallace’s post didn’t work for you, try this.
I don’t understand what your statement means in the presence of indexical uncertainty, and anyway it doesn’t seem to be a correct summary of Stuart’s idea. He only takes the average within each branch where all kids get the same outcome anyway, not the average between branches. The averaging here is justified, not by referring to indexical probabilities (which we don’t understand), but by noting that it preserves the utility obtained by each agent in each possible world. After that Stuart uses UDT (not another layer of averaging, which would be unjustified) to obtain the “correct” decision in the aggregate for all branches. Reread his solution to model 4 and then model 1 which it refers to—the reasoning is quite tricky.
Ah, sorry, I’m not very familiar with UDT. Avoiding indexical probabilities (they aren’t that poorly understood) seems to be at the root of the unintuitive answer you cite, though: would you still give your world being world 1 a probability of 1⁄2 if there were 10000 kids in world 2?
What use is my intuitive answer to you? We’re trying to figure out what the correct answer should be, not holding an opinion poll.
What use is intuition to anyone? Since you mentioned your intuitive reaction to the UDT answer in your post, your intuition seems to have at least a little credibility.
I could talk about the logical reasons to assign greater chance of being in circumstances with more people, if you’d prefer.
I think that would be counterproductive too, unless you have already reviewed the previous discussions of anthropics on LW and are certain that your argument is new.
We don’t understand much about indexicals right now, but we do understand that questions about subjective probabilities should be reformulated as decision problems, otherwise you get confused on easy problems like Sleeping Beauty and never get to the challenging stuff. The reason: we all agree now that decision problems ask which precommitment would be viewed by the agent as most beneficial (Eliezer’s intuition behind TDT), and precommitments take place before you look at any evidence, indexical or otherwise.
Do you feel that talking about probabilities instead of decisions brings new clarity to our problem? Then explain.
ETA: maybe rwallace’s post The I-Less Eye will be helpful to you.
Well, both the numbers and the logical steps required are identical in this case, no matter whether you think about it as decision theory or probability. The reason Sleeping Beauty is tricky is that assigning probabilities the obvious way leads to breaking normalization, but that’s not a problem in simple cases like this.
Either way, you should still use the fact that you have an equal chance of being any person whose circumstances fit yours, lacking any other information (this can be shown to be the correct prior, following from the principle of indifference, which says that if the information in a case is identical, the probability should be too).
Sorry, what’s a “correct prior”?
Also, this post might change your mind about the merits of subjective probabilities vs decision problems, even if rwallace’s post failed to.
Correct as in uniquely fulfills the desiderata of probability theory, on which the whole thing can be based. Ooh, found a link, I didn’t know that was online. Particularly important for these purposes is the principle that says that states with identical information should be assigned identical probabilities. You just know that you are one of the people in the problem, which breaks the symmetry between the two coin-flip outcomes (since there are different numbers of people depending on the outcome), but it creates a symmetry between all the states specified by “you are one of the people in this problem.”
It’s not that I’m saying it’s wrong to approach this as a decision problem. Just that ordinary probability applies fine in this case. If you get a different result with a decision theory than with bayesian probability, though, that is bad. Bad in the sense of provably worse by the measure of expected utility, unless circumstances are very extreme.
We’re still talking past each other, I’m afraid.
What’s “expected utility” in situations with indexical uncertainty? If you take the “expectation” according to an equal weighting of all indistinguishable observer-moments, isn’t your reasoning circular?
Also I’m interested in hearing your response to rwallace’s scenario, which seems to show that assigning equal probabilities to indistinguishable observer-moments leads to time-inconsistency.
Ah, wait: the probability in rwallace’s post was 1⁄99 all along. This is because each time you do the “clone me” operation, it’s not like it automatically halves the probability; it only works that way for a very certain set of evidence. When you are careful and specify which evidence is available, the apparent problem is resolved.
I’m afraid I can’t take your word for that, please show me the calculations.
Hmm, actually I might be making the wrong correction since this would contradict the rule that P(AB) = P(A)*P(B|A). But my plan was to specify the “anthropic evidence” (memories, body, etc.) as being exactly the same stuff that makes you “you” at the start of the process, then clarifying the question as P(original | anthropic evidence).
Upon reflection, this is very shaky, but still possibly correct. I’ll try and formalize the change to the product rule and see what it says about Sleeping Beauty, which has a relatively known answer.
This still doesn’t look like the calculations that I asked for… ?
Oh, sorry; the calculations are trivial. It’s the parts that aren’t math that are the problem.
Take a person imagining whether to get copied 98 times. He wants to know “after I get out of the machine but before I get told if I’m the original, what is the probability that I am the orginial?” There are two different “edge cases.”
1) If during the copying process you generate no new evidence, i.e. all your copies get near-identical memories- if that’s the case, then the principle of indifference applies with overwhelming obviousness. You have nothing with which to differentiate yourself, so you must assign an equal probability, so the probability that you’re the original is 1⁄99.
2) You have to go in in 98 different sessions to get copied, thus generating extra evidence (memories) in between each time. Here, the only place you can apply the principle of indifference is within each session, so you have a probability of 1⁄2 of being the original after every session. The product rule then says that since P(AB) = P(A)*P(B|A), your final probability of being the original is (1/2)^98. But this feels odd because at no time during the process could this probability be realized—when waking up you always have a probability estimate of 1⁄2 - rendering this immune from the sort of betting game you might play in e.g. the Sleeping Beauty problem.
With these two cases in mind, we can consider a copying scheme with the causal structure of (2) but the evidence of (1) (copied in series, but no distinguishing memories). Logic would say that the evidence wins, since the evidence usually wins. But either the product rule stops working or the probability P(B|A) changes in an unusual way that must exactly parallel what the evidence requires—this provides a license for just using the evidence and not using the product rule in these sorts of cases, but it would be interesting to see whether it’s possible to save the product rule.
Saving the product rule would require changing the P(B|A) in P(AB)=P(A)*P(B|A). If we just look at the case where you’re copied 2 times in a row with identical memories, A = original the first time and B = original the second time. One woudl expect that P(A) = 1⁄2. But P(AB)=1/3, so P(B|A) must equal 3⁄4. For 98 copies the last conditional probability would be 97/98! This is really weird. So I’m still thinking about it.
Hey, I just had this idea. To get the required values of P(B|A) you may try to consider the possibility that the “subjective continuation” of a copy (which was made from the original) can jump into another copy (also made from the original, not from the first copy). There seems to be no apriori reason why that shouldn’t happen, if the information states of the copies are equivalent. Why focus on the physical continuity within the copying process anyway? Information is all that matters.
This way you get to keep the illusion that you “are” one specific copy at all times, rather the set of all identical information states at once (my preferred point of view up till now). I wonder if some other thought experiment could break that illusion more conclusively.
Hm, I guess it is circular. Dang. The question is really “is bayesian probability correct in general?”
Do you mean rwallace’s scenario with the copies? The probabilities seem correct, though since there are multiple copies, normalization might be broken (or artificially enforced) somewhere I didn’t notice, like in Sleeping Beauty. I’m a bit unsure. What is clear is that there isn’t actually a discontinuity at short times—since the probability comes form the evidence of your memories, not how it “really happened.”
EDIT: There does appear to be an inconsistency when making serial copies—computing it different ways gives different answers. Freaking normalization.
So if I have a biased coin that comes up heads 90% of the time, flip it and hide the result from you, your credence for heads should be 50%? Nah. I’m willing to believe in the principle of indifference only if the cases are completely identical, not just observably identical. In our problem the cases aren’t completely identical—you may be the sole child or one of 10 children, and these correspond to very different and non-symmetrical states of the world.
It seems to me that you’re confused. Sleeping Beauty isn’t tricky at all, but our problem is. If rwallace’s post didn’t work for you, try this.