In the selfish sleeping beauty case, assuming the incubator variant, what does your solution say a Beauty should do if we tell her that she is in Room 1 and ask her what price she would pay for a lottery ticket that pays $1 on Heads?
Selfish agents have problems with precommitments. When you tell them “you are in Room 1”, this is not only changing their information, but also changing their utility. To wit:
Before being told where they are, selfish and average utilitarians have identical preferences over outcomes. When they consider the potential “if I was to be told I was in room 1 and offered that lottery ticket for $x, would I take it”, they calculate the expected utility of doing so as 1-x in the heads world and -x/2 in the tails world (from the selfish point of view, “in the tails world, there’s a 50% chance I won’t have to pay, even if I commit to accepting”), hence 1-3/2x in total. So she would precommit to taking the deal if x<2/3.
However, after being told that she is in room 1, the selfish SB’s preferences change: her previous utility in the tails world was the average of the utilities of the SBs in room 1 and 2, but now it is entirely the utility of the SB in room 1. Her expected utility is now 1-x -x, so she would only want to take it for x<1/2.
So, if she sticks to her precommitments, she would accept x<2/3; if she breaks her precommitments (leaving her possibly money-pumpable), she would only accept x<1/2.
Suppose we pull 100 people off the street. We throw a coin and tell 99 subjects how coin actually landed, and tell the remaining subject the opposite. Suppose you hear “heads”. How much would you (assuming you’re a selfish agent) pay for a ticket that pays $1 on heads?
The intuitive answer is <$0.99, but section 3.3.3 says the answer should be <$0.50. To generalize this example, an agent following section 3.3.3 never “updates” (in the sense of changing their betting odds) on any evidence unless the evidence allows them to rule out being in some possible world, but since we appear to live in an infinite universe (so everything that can happen does happen somewhere), such an agent would never update.
Since section 3.3.3 is both reflectively inconsistent and highly counter-intuitive even in simple straightforward cases, I feel fairly confident that it’s not the right solution to “selfish preferences”. I think a better direction to look is UDASSA (or AIXI, which treats anthropic reasoning in a similar way), which is more SIA-like. I still don’t know how to deal with copying or death (i.e., issues related to quantum immortality) but at least they don’t run into so much trouble on these simpler problems.
The intuitive answer is <$0.99, but section 3.3.3 says the answer should be <$0.50
? I don’t see this at all.
By section 3.3.3, I assume you mean the isomorphism between selfish and average-utilitarian? From an average utilitarian perspective (which is the same as a total utilitarian for fixed populations), buying that ticket for x after hearing “heads” will lose one person x in the tails world, and gain 99 people 1-x in the heads world. So the expected utility is (1/2)(1/100)(-x+99(1-x)), which is positive for x< 99⁄100.
ADT is supposed to reduce to a simplified version of UDT in non-anthropic situations; I didn’t emphasise this aspect, as I know you don’t want UDT published.
Section 3.3.3 says that a selfish agent should make the same decisions as an average-utilitarian who averages over just the set of people who may be “me”, right? That’s why it says that in the incubator experiment, a selfish agent who has been told she is in Room 1 should pay 1⁄2 for the ticket. An average-utilitarian who averages over everyone who exists in a world would pay 2⁄3 instead.
So in my example, consider an average-utilitarian whose attention is restricted to just people who have heard “heads”. Then buying a ticket loses an average of x in the tails world, and gains an average of 1-x in the heads world, so such an restricted-average-utilitarian would pay x<1/2.
(If this is still not making sense, please contact me on Google Chat where we can probably hash it out much more quickly.)
We’ll talk on google chat. But my preliminary thought is that if you are indeed restricting to those who have heard heads, then you need to make use of the fact that this objectively much more likely to happen in the heads world than in the tails.
A good question from Wei Dai:
Selfish agents have problems with precommitments. When you tell them “you are in Room 1”, this is not only changing their information, but also changing their utility. To wit:
Before being told where they are, selfish and average utilitarians have identical preferences over outcomes. When they consider the potential “if I was to be told I was in room 1 and offered that lottery ticket for $x, would I take it”, they calculate the expected utility of doing so as 1-x in the heads world and -x/2 in the tails world (from the selfish point of view, “in the tails world, there’s a 50% chance I won’t have to pay, even if I commit to accepting”), hence 1-3/2x in total. So she would precommit to taking the deal if x<2/3.
However, after being told that she is in room 1, the selfish SB’s preferences change: her previous utility in the tails world was the average of the utilities of the SBs in room 1 and 2, but now it is entirely the utility of the SB in room 1. Her expected utility is now 1-x -x, so she would only want to take it for x<1/2.
So, if she sticks to her precommitments, she would accept x<2/3; if she breaks her precommitments (leaving her possibly money-pumpable), she would only accept x<1/2.
Suppose we pull 100 people off the street. We throw a coin and tell 99 subjects how coin actually landed, and tell the remaining subject the opposite. Suppose you hear “heads”. How much would you (assuming you’re a selfish agent) pay for a ticket that pays $1 on heads?
The intuitive answer is <$0.99, but section 3.3.3 says the answer should be <$0.50. To generalize this example, an agent following section 3.3.3 never “updates” (in the sense of changing their betting odds) on any evidence unless the evidence allows them to rule out being in some possible world, but since we appear to live in an infinite universe (so everything that can happen does happen somewhere), such an agent would never update.
Since section 3.3.3 is both reflectively inconsistent and highly counter-intuitive even in simple straightforward cases, I feel fairly confident that it’s not the right solution to “selfish preferences”. I think a better direction to look is UDASSA (or AIXI, which treats anthropic reasoning in a similar way), which is more SIA-like. I still don’t know how to deal with copying or death (i.e., issues related to quantum immortality) but at least they don’t run into so much trouble on these simpler problems.
? I don’t see this at all.
By section 3.3.3, I assume you mean the isomorphism between selfish and average-utilitarian? From an average utilitarian perspective (which is the same as a total utilitarian for fixed populations), buying that ticket for x after hearing “heads” will lose one person x in the tails world, and gain 99 people 1-x in the heads world. So the expected utility is (1/2)(1/100)(-x+99(1-x)), which is positive for x< 99⁄100.
ADT is supposed to reduce to a simplified version of UDT in non-anthropic situations; I didn’t emphasise this aspect, as I know you don’t want UDT published.
Section 3.3.3 says that a selfish agent should make the same decisions as an average-utilitarian who averages over just the set of people who may be “me”, right? That’s why it says that in the incubator experiment, a selfish agent who has been told she is in Room 1 should pay 1⁄2 for the ticket. An average-utilitarian who averages over everyone who exists in a world would pay 2⁄3 instead.
So in my example, consider an average-utilitarian whose attention is restricted to just people who have heard “heads”. Then buying a ticket loses an average of x in the tails world, and gains an average of 1-x in the heads world, so such an restricted-average-utilitarian would pay x<1/2.
(If this is still not making sense, please contact me on Google Chat where we can probably hash it out much more quickly.)
We’ll talk on google chat. But my preliminary thought is that if you are indeed restricting to those who have heard heads, then you need to make use of the fact that this objectively much more likely to happen in the heads world than in the tails.