You provide very little information. I’m not even sure what you disagree with exactly. If it would be inconvenient for you to explain your disagreement that’s fine, but I’m didn’t update much on your comment. If you want to give me a bit more information about your state of mind, you can tell me how familiar you are with UDT and whether you think it is on the right path towards solving anthropics.
Yeah, sorry about that. The basic idea is that by providing multiple answers, the proposal has immediately given up on getting the same answer as an expected utility maximizer. This is a perfectly fine thing to do if probabilities are impossible to assign, and so maximizing expected utility breaks down. But probability does not in fact break down when confronted with anthropic situations, so picking from N answers just gives you at least an (N-1)/N chance of being wrong.
Expected utility does break down in the presence of indexical uncertainty though; if there are multiple agents with exactly your observations, it is important to take into account that your decision is the one they will all make. Psy-Kosh’s non-anthropic problem deals with this sort of thing, though it also points out that such correlation between agents can exist even without indexical uncertainty, which is irrelevant here.
I’m not sure what the N answers that you are talking about are. The different solutions in Stuart’s paper refer to agents with different utility functions. Changing the utility function usually does change the optimal course of action.
Psy-Kosh’s non-anthropic problem is just regular uncertainty. The experimenters flipped a coin, and you don’t know if the coin is heads or tails. The collective decision making then runs you into trouble. I can’t think of any cases with indexical uncertainty but no collective decision making that run into similar trouble—in the Sleeping Beauty problem at least the long-term frequency of events is exactly the same as the thing you plug into the utility function to maximize average reward, unlike in the non-anthropic problem. Do you have an example you could give?
EDIT: Oh, I realized one myself—the absent-minded driver problem. In that problem, if you assign utility to the driver at the first intersection—rather than just the driver who makes it to an exit—you end up double-counting and getting the wrong answer. In a way it’s collective decision-making with yourself—you’re trying to take into account how past-you affected present-you, and how present-you will affect future-you, but the simple-seeming way is wrong. In fact, we could rejigger the problem so it’s a two-person, non anthropic problem! Then if we do the reverse transform on Psy-kosh’s problem, maybe we could see something interesting… Update forthcoming, but the basic idea seems to be that the problem is when you’re cooperating with someone else, even yourself, and are unsure who’s filling what role. So you’re pretty much right.
The objects in Stuart’s paper are decision procedures, but do not involve utility directly (though it is a theorem that you can find a utility function that gives anything). Utility functions have to use a probability before you get a decision out, but these decision procedures don’t. Moreover, he uses terms like “average utilitarian” to refer to the operations of the decision procedure (averages individual utilities together), rather than the properties of the hypothetical corresponding utility function.
What’s happening is that he’s taking an individual utility function and a decision procedure, and saying that together these specify what happens. And I’m saying that this is an over-specified problem.
Then if we do the reverse transform on Psy-kosh’s problem, maybe we could see something interesting...
That’s what I was originally trying to suggest, but it seems I was unclear. The absent-minded driver is a simpler example anyways, and does deal with exactly the kind of breakdown of expected utility I was referring to.
What’s happening is that he’s taking an individual utility function and a decision procedure, and saying that together these specify what happens. And I’m saying that this is an over-specified problem.
Each decision procedure is derived from a utility function. From the paper:
Anthropic Decision Theory (ADT) An agent should first find all the decisions linked with their own. Then they should maximise expected utility, acting as if they simultaneously controlled the outcomes of all linked decisions, and using the objective (non-anthropic) probabilities of the various worlds.
This fully specifies a decision procedure given a utility function. There is no second constraint taking the utility function into account again, so it is not overspecified.
Simpler? Hm. Well, I’m still thinking about that one.
Anyhow, by over-specified I mean that ADT and conventional expected-utility maximization (which I implicitly assumed to come with the utility function) can give different answers. For example, in a non-cooperative problem like copying someone either once or 10^9 times, and then giving the copy a candybar if it can correctly guess how many of them there are. The utility function already gives an answer, and no desiderata are given that show why that’s wrong—in fact, it’s one of the multiple possible answers laid out.
Simpler in that you don’t need to transform it before it is useful here.
Standard expected utility maximization requires a probability distribution, but the problem is that in anthropic scenarios it is not obvious what the correct distribution is and how to correctly update it. ADT uses the prior distribution before ‘observing one’s own existence’, so it circumvents the need to preform anthropic updates.
I’m not sure which solution to your candybar problem you think is correct because I am not sure which probability distribution you think is correct, but all the solutions in the paper that disagree with yours actually are what you would want to precommit to given the associated utility function and are therefore correct.
Standard expected utility maximization requires a probability distribution, but the problem is that is anthropic scenarios it is not obvious what the correct distribution is and how to correctly update it.
If it was solved in a way that made it obvious for, say, the Sleeping Beauty problem, would that then be the right way to do it?
all the solutions in the paper that disagree with yours actually are what you would want to precommit to given the associated utility function and are therefore correct.
I think you’re just making up utility functions here—is a real utility function (that is, a function of the state of the world) ever calculated in the paper, other than the use of the individual utility function? And we’re talking about regular ol’ utility functions, why are ADT’s decisions necessarily invariant under changing time-like uncertainty (normal sleeping beauty problem) to space-like uncertainty (sleeping beauty problem with duplicates)?
If it was solved in a way that made it obvious for, say, the Sleeping Beauty problem, would that then be the right way to do it?
I would tentatively agree. To some extent the problem is one of choosing what it means for a distribution to be correct. I think that this is what Stuart’s ADT does (though I don’t think it’s a full solution to this).
You would also still need to account for acausal influence. Just picking a satisfactory probability distribution doesn’t ensure that you will one box on Newcomb’s problem, for example.
I think you’re just making up utility functions here—is a real utility function (that is, a function of the state of the world) ever calculated in the paper, other than the use of the individual utility function?
Is this quote what you had in mind? It seems like calculating a utility function to me, but I’m not sure what you mean by “other than the use of the individual utility function”.
In the tails world, future copies of myself will be offered the same deal twice. Any profit they make will be dedicated to hugging orphans/drowning kittens, so from my perspective, profits (and losses) will be doubled in the tails world. If my future copies will buy the coupon for £x, there would be an expected £0.5(2 × (−x + 1) + 1 × (−x + 0)) = £(1 − 3/2x) going towards my goal. Hence I would want my copies to buy whenever x < 2⁄3.
That is from page 7 of the paper.
And we’re talking about regular ol’ utility functions, why are ADT’s decisions necessarily invariant under changing time-like uncertainty (normal sleeping beauty problem) to space-like uncertainty (sleeping beauty problem with duplicates)?
They’re not necessarily invariant under such changes. All the examples in the paper were, but that’s because they all used rather simple utility functions.
And if we’re talking about regular ol’ utility functions, why are ADT’s decisions necessarily invariant under changing time-like uncertainty (normal sleeping beauty problem) to space-like uncertainty (sleeping beauty problem with duplicates)?
They’re not necessarily invariant under such changes. All the examples in the paper were, but that’s because they all used rather simple utility functions.
Hm, yes, you’re right about that.
Anyhow, I’m done here—I think you’ve gotten enough repetitions of my claim that if you’re not using probabilities, you’re not doing expected utility :) (okay, that was an oversimplification)
You provide very little information. I’m not even sure what you disagree with exactly. If it would be inconvenient for you to explain your disagreement that’s fine, but I’m didn’t update much on your comment. If you want to give me a bit more information about your state of mind, you can tell me how familiar you are with UDT and whether you think it is on the right path towards solving anthropics.
Yeah, sorry about that. The basic idea is that by providing multiple answers, the proposal has immediately given up on getting the same answer as an expected utility maximizer. This is a perfectly fine thing to do if probabilities are impossible to assign, and so maximizing expected utility breaks down. But probability does not in fact break down when confronted with anthropic situations, so picking from N answers just gives you at least an (N-1)/N chance of being wrong.
Expected utility does break down in the presence of indexical uncertainty though; if there are multiple agents with exactly your observations, it is important to take into account that your decision is the one they will all make. Psy-Kosh’s non-anthropic problem deals with this sort of thing, though it also points out that such correlation between agents can exist even without indexical uncertainty, which is irrelevant here.
I’m not sure what the N answers that you are talking about are. The different solutions in Stuart’s paper refer to agents with different utility functions. Changing the utility function usually does change the optimal course of action.
Psy-Kosh’s non-anthropic problem is just regular uncertainty. The experimenters flipped a coin, and you don’t know if the coin is heads or tails. The collective decision making then runs you into trouble. I can’t think of any cases with indexical uncertainty but no collective decision making that run into similar trouble—in the Sleeping Beauty problem at least the long-term frequency of events is exactly the same as the thing you plug into the utility function to maximize average reward, unlike in the non-anthropic problem. Do you have an example you could give?
EDIT: Oh, I realized one myself—the absent-minded driver problem. In that problem, if you assign utility to the driver at the first intersection—rather than just the driver who makes it to an exit—you end up double-counting and getting the wrong answer. In a way it’s collective decision-making with yourself—you’re trying to take into account how past-you affected present-you, and how present-you will affect future-you, but the simple-seeming way is wrong. In fact, we could rejigger the problem so it’s a two-person, non anthropic problem! Then if we do the reverse transform on Psy-kosh’s problem, maybe we could see something interesting… Update forthcoming, but the basic idea seems to be that the problem is when you’re cooperating with someone else, even yourself, and are unsure who’s filling what role. So you’re pretty much right.
The objects in Stuart’s paper are decision procedures, but do not involve utility directly (though it is a theorem that you can find a utility function that gives anything). Utility functions have to use a probability before you get a decision out, but these decision procedures don’t. Moreover, he uses terms like “average utilitarian” to refer to the operations of the decision procedure (averages individual utilities together), rather than the properties of the hypothetical corresponding utility function.
What’s happening is that he’s taking an individual utility function and a decision procedure, and saying that together these specify what happens. And I’m saying that this is an over-specified problem.
That’s what I was originally trying to suggest, but it seems I was unclear. The absent-minded driver is a simpler example anyways, and does deal with exactly the kind of breakdown of expected utility I was referring to.
Each decision procedure is derived from a utility function. From the paper:
This fully specifies a decision procedure given a utility function. There is no second constraint taking the utility function into account again, so it is not overspecified.
Simpler? Hm. Well, I’m still thinking about that one.
Anyhow, by over-specified I mean that ADT and conventional expected-utility maximization (which I implicitly assumed to come with the utility function) can give different answers. For example, in a non-cooperative problem like copying someone either once or 10^9 times, and then giving the copy a candybar if it can correctly guess how many of them there are. The utility function already gives an answer, and no desiderata are given that show why that’s wrong—in fact, it’s one of the multiple possible answers laid out.
Simpler in that you don’t need to transform it before it is useful here.
Standard expected utility maximization requires a probability distribution, but the problem is that in anthropic scenarios it is not obvious what the correct distribution is and how to correctly update it. ADT uses the prior distribution before ‘observing one’s own existence’, so it circumvents the need to preform anthropic updates.
I’m not sure which solution to your candybar problem you think is correct because I am not sure which probability distribution you think is correct, but all the solutions in the paper that disagree with yours actually are what you would want to precommit to given the associated utility function and are therefore correct.
If it was solved in a way that made it obvious for, say, the Sleeping Beauty problem, would that then be the right way to do it?
I think you’re just making up utility functions here—is a real utility function (that is, a function of the state of the world) ever calculated in the paper, other than the use of the individual utility function? And we’re talking about regular ol’ utility functions, why are ADT’s decisions necessarily invariant under changing time-like uncertainty (normal sleeping beauty problem) to space-like uncertainty (sleeping beauty problem with duplicates)?
I would tentatively agree. To some extent the problem is one of choosing what it means for a distribution to be correct. I think that this is what Stuart’s ADT does (though I don’t think it’s a full solution to this).
You would also still need to account for acausal influence. Just picking a satisfactory probability distribution doesn’t ensure that you will one box on Newcomb’s problem, for example.
Is this quote what you had in mind? It seems like calculating a utility function to me, but I’m not sure what you mean by “other than the use of the individual utility function”.
That is from page 7 of the paper.
They’re not necessarily invariant under such changes. All the examples in the paper were, but that’s because they all used rather simple utility functions.
Hm, yes, you’re right about that.
Anyhow, I’m done here—I think you’ve gotten enough repetitions of my claim that if you’re not using probabilities, you’re not doing expected utility :) (okay, that was an oversimplification)