Dutch-book resistant probability over centered worlds

[jessicata]

Link post

An uncentered world is an objective state-trajectory of the material universe; I ignore quantum complications. If you know the uncentered world, it does not follow that you can predict your proximate observations, since you do not know which part of the uncentered world is here and now. A centered world is an uncentered world combined with a “here and now” tag for “where am I /​ what time is it”.

I examine consistent probability assignments over centered worlds, which have relevance to anthropics. The main assumption I make is that these probabilities should not be Dutch-bookable if used by a CDT agent. Dutch book arguments (e.g. diachronic Dutch book arguments for Bayesian updating) typically assume CDT in the background; it is not straightforward to work out which bets EDT will accept in general. CDT Dutch book resistance therefore provides a normative probability framework that generalizes arguments for Bayesian probability.

Thought experiments such as Sleeping Beauty, and variants involving duplication, question how to assign probabilities to centered worlds in situations involving memory loss. One can analogize memory loss to being an individual who is part of a collective with shared goals; such an individual would be motivated to use probabilities over their centered world to take actions towards the shared objective, in a way consistent with how other individuals in the same collective do so. (In the case of AI, it is easier to see how the line between “memory loss” and “member of collective” is not very decision-theoretically important, as the same hardware can run many episodes in sequence with different observations, and both “memory loss” and “member of collective” interpretations are possible.)

What I will show is that the probability assignments that resist Dutch books are those which are “centering-uniform” in that they are arrived at by starting with some fixed measure over uncentered worlds, and weighting them according to number of occurrences of the agent’s information state, with uniform probability over centers which have the same world and information state, and with any probabilities allowed conditional on information states with zero -measure. This condition strongly resembles Bostrom’s SSSA (strong self-sampling assumption) and/​or SSSA+SIA, which only disagree with each other up to a population scaling factor on uncentered worlds. Moreover, as shown in the appendix, when a probability policy resists weak Dutch books, it is centering-uniform with respect to a measure which is non-dogmatic in assigning non-zero measure to all populated uncentered worlds; this “strict centering-uniformity” enforces more extensive agreement with SSSA(+SIA).

Mathematical formulation

Let’s set up the finite case formally. Let W be a finite set of (uncentered) worlds. Let C be a finite set of centers (roughly, possible observer-moments, though different time grains can be taken). Let “decenter” a centered world into its uncentered world. Let I be a finite set of information states. Let “observe” a centered world, yielding an information state possessed by the agent with that center. To simplify, we assume o is surjective.

A probability policy assigns to each information state some probability distribution over centers. This represents the probabilities an observer assigns to centered worlds upon making their observation i. We assume all probability policies have whenever ; in other words, is only supported on .

A betting menu gives an agent in information state i the option of buying a contract which pays out in case their center is c (noting, we must have ). A betting menu b is acceptable to a probability policy q iff CDT with q probabilities endorses accepting it in all information states; formally, . (We imagine, for simplicity, that accepting /​ rejecting the bet makes no difference to the universe other than bank balances, and that the agent’s utility is linear in money. This avoids more complicated self-ratification issues; see “On the Interpretation of Decision Problems with Imperfect Recall” and “The Computational Complexity of Single-Player Imperfect-Recall Games” for details on situations where decisions can affect the trajectory. We imagine that all instances of agents with varying centers have the common objective of maximizing the bank balance; we could pretend that this balance is dedicated to an altruistic fund.)

In uncentered world w, the total payoff if all bets are accepted is . Let be the set of populated worlds. A Dutch book for a probability policy q is a betting menu b which is acceptable to q and which results in a sure loss in populated worlds, in the sense that . We wish to characterize probability policies which are not Dutch-bookable.

Say a probability policy q is centering-uniform iff for some probability distribution over with , we have, for all and ,

.

When the sum is zero, is unconstrained. In the case where the sum is non-zero, this is a more familiar SSSA+SIA expression:

A “uniformity” implication is that if have the same uncentered world w with and the same information state i, then , in agreement with SSSA. Note that we can move between SSSA and SSSA+SIA by changing to adjust for population. Hence, centering-uniformity is consistent with both.

The main result: A probability policy is Dutch-bookable iff it is not centering-uniform.

Proof:

A betting menu is a vector . Define the matrix . Define the matrix , .

Now is the CDT expected value of accepting at i, and is the payoff in populated uncentered world w. A Dutch book corresponds to b which satisfies and , componentwise.

Due to Motzkin’s transposition theorem, exactly one of the following systems has a solution:

1. Find with and .

2. Find and with .

Problem (1) has a solution iff there is a Dutch book for q. Problem (2) has a solution iff it has a solution with being a probability distribution over populated worlds (by re-scaling); call problem (2) with this extra constraint (2p). Note for all ,

.

First, suppose (2p) has a solution. Then for all , . Pick any and sum this equation over :

Since , we have . So for all ,

.

This shows q to be centering-uniform (extend with for un-populated w). So any non-Dutch-bookable probability policy is centering-uniform.

In the other direction, suppose q is centering-uniform as witnessed by . Then and provide a solution to (2), showing that q is not Dutch-bookable.

Application to Sleeping Beauty

We set up Sleeping Beauty as follows. The centered worlds under consideration are HS (Heads on Sunday), TS (Tails on Sunday), HM (Heads on Monday), TM (Tails on Monday), TT (Tails on Tuesday). It is of course possible to add more (e.g. for after the experiment); this is a minimal setup. The uncentered worlds are H, T (for Heads, Tails). The information states are S, MT (for Sunday and not knowing the coin; and for unknown Monday/​Tuesday and not knowing the coin, the awakening state).

A probability policy q will assign to each information state in {S, MT} some probability distribution over centered worlds. By assumption is only supported on {HS, TS} and is only supported on {HM, TM, TT}. Now q is Dutch-book resistant iff it is centering-uniform, i.e. iff for some probability distribution on {H, T}, we have:

Generally, both “halfers” and “thirders” agree that , i.e. on Sunday, Beauty should assign 50% to Heads. Under this supposition, centering-uniformity forces thirding: . However, centering-uniformity is compatible with by setting . This reflects the fact that centering-uniformity is agnostic about the measure on uncentered worlds.

A non-dogmatic probability policy (which has ) must predictably change its probability of Heads from Sunday to Monday, despite the absence of a memory wipe during Sunday night. Many (such as David Lewis) have found this predictable change in probabilities counter-intuitive, but Dutch book analysis suggests that such a predictable change is rationally obligatory. For more on Dutch book arguments applied to Sleeping Beauty, see “Simple Dutch books for Sleeping Beauty halfers”.

Conclusion

This post presents a framework for Dutch-book resistant probability assignments in cases of amnesia, which can be applied to anthropic situations. The basic framework is Bostrom’s SSSA, and scaling factors such as SIA are optional. The uncentered measure is a representational parameter for Dutch-book resistant probability policies, which are equivalently centering-uniform probability policies. need not be interpreted as the “prior over universes” in standard anthropics (SSA /​ SIA). Rather, parameterizes the Dutch-book resistant probability policies, analogous to the subjective prior in ordinary Bayesianism, which parameterizes Dutch-book resistant ways of assigning posterior probabilities.

To the degree Dutch book arguments for Bayesianism succeed, so do Dutch book arguments for centering-uniformity in settings with amnesia. Possible escape routes, such as EDT, could also be escape routes from ordinary Dutch book arguments for Bayesianism. Accordingly, centering-uniformity is a normative generalization of Bayesian probability to amnesic situations, since it stands or falls by similar arguments. See “Can de se choice be ex ante reasonable in games of imperfect recall? A complete analysis” for more on the EDT+FNC escape route, which diverges from Bayesian updating.

Philosophically, this post’s conclusions are contrary to those of Emily Adlam’s “Against Self-Location”; while Adlam argues that there are no rationality constraints for agents’ conditional probabilities on their center given their uncentered world (“pure self-locating credences”), I argue that there are, at least under CDT Dutch-book assumptions used to justify Bayesianism; and if anything, pure self-locating credences are more rationally constrained than non-self-locating, -dependent credences over uncentered worlds are. Adlam argues that self-locating uncertainty and indexical caring measure can be ambiguous for agents who are indexically selfish; while I agree in the pure case of indexical selfishness, I argue that these are disambiguated for -altruistic agents who use self-locating uncertainty for decisions that affect the altruistic goals.

The most straightforward mathematical extension possible is to infinite sets of worlds and centers. In particular, and could be restricted (e.g. regular Borel) probability measures over infinite sample spaces. Tools from functional analysis, such as the Riesz-Markov representation theorem, might apply usefully; betting menus could be continuous functions of the space of centers (given some topology), and probability assignments could assign subjective expected values to such menus. I’ve concentrated on the finite case in this post because it is much more straightforward.

Appendix: weak Dutch books

We examine resistance to weak Dutch books, which guarantee no gain for the agent, and make loss possible. A weak Dutch book for q is a betting menu b which is acceptable to q, for which for all , and for which for some .

Call a probability policy q strictly centering-uniform iff it is centering-uniform for some which assigns non-zero probability to all populated worlds. We show: q is strictly centering-uniform iff there are no weak Dutch books against q.

Proof:

We construct the same matrices as before and consider the two problems:

1. Find with .

2. Find , , and with , , , .

By Motzkin’s transposition theorem, exactly one of these has a solution. The first has a solution iff there is a weak Dutch book for q. If the second has a solution, we set and get a solution to the problem (2) of the main proof, showing that q is strictly centering-uniform. Furthermore, if q is strictly centering-uniform (with measure ), we get a solution to (2) with , , and , which shows there are no weak Dutch books against q.

Strict centering-uniformity is analogous to a requirement in ordinary Bayesianism to assign positive probability to every possible world in the sample space, at least in the finite case. It is a non-dogmatism condition.

[cousin_it]

This is tricky. What do we want to use probabilities for?

If you talk about Dutch books, then you’re viewing probabilities instrumentally, as a tool to make decisions. But we already know they can’t be used for that, because of situations where the decision you’re about to make affects the number of copies of the mind-state making the decision. The simplest example of such problem is the Absent-Minded Driver. The optimal decision there is obtained by UDT reasoning, not by taking a probability distribution over where you are and optimizing from that. Since there’s nothing stopping the world from behaving at least a tiny bit like that problem, the whole direction of justifying probabilities by decisions is probably a dead end.

Alternatively, if you view probabilities as “reality fluid” that determines what to expect in the next moment, that’s an important question too, but it’s not clear why Dutch books help solve it.

This whole thing is one of those LWish puzzles where we figured out a bunch of things very quickly, then ran into a wall and have been stuck for over a decade since. I’d really love to see some crack in the wall.

[jessicata]

So the whole direction of justifying probabilities by decisions is probably a dead end.

CDT+SSSA+SIA gives KKT conditions, covers all UDT policies in absent-minded games, is ex ante optimal in additive games.… A bit more discussion in other post. The part where number of copies is affected does not prevent CDT+SSSA+SIA from being well-defined (in the sense of having self-ratifying probability /​ policy combinations), or from giving ex ante utility stationarity conditions. It just complicates the scenario, which is why I didn’t focus on it in this post.

Alternatively, if you view probabilities as “reality fluid” that determines what to expect in the next moment, that’s an important question too, but it’s not clear why Dutch books help solve it.

This seems more in metaphysics territory. I take the physicalist baseline to be that one should assign probabilities to centered worlds and that’s all. Maybe there is other stuff to assign probabilities to (like qualia, idk). I think before assigning numbers a thing to do is to figure out what is the metaphysical status (if any) of the entity being assigned probabilities. Since it does not seem clear to me that there is a there there. (Whereas with things like the Sleeping Beauty problem or Doomsday Argument it seems there is plenty of disagreement to be had about rational probabilities on centered worlds, even without getting into any metaphysics on top of centered worlds)

[cousin_it]

Thank you for the links! I’m not sure they fully answer my question though. It’s more about what we’re trying to do.

1. Maybe we want to start with probabilities and derive the optimal policy, continuing the project of VNM utility maximization. To me that project basically died with Absent-Minded-Driver, because it shows we can’t start with probabilities. Having “self-ratifying probability/​policy combinations” doesn’t have quite the same pull. And in any case the necessity of self-coordination (UDT1.1) forces us to choose whole policies, not individual actions. Or do you hope that if we push probability-based approaches far enough, they can close the gap?

2. Or maybe we first choose which policy to follow, then compute the probabilities of finding ourselves at one node or another. That’s fine, and I do think SIA is the best answer. And maybe even we can show that these probabilities have nice decision-making properties, but to me this seems like an “echo” of us choosing the best policy to begin with, no?

Or maybe I’m being silly again and misunderstanding the whole thing?

[jessicata]

Primarily 2, with the extra bit where we can be assured there’s a CDT+SIA fixed point. For the purpose of anthropic probabilities, that’s a desirable condition. We don’t need probabilities to force optimality but we would rather they be compatible with optimality, and preferably that they help with the computation.

The other part is that UDT is not really computationally tractable and so figuring out how to optimize it is useful, which is more (1) territory. For this the computational complexity paper is relevant because it shows CDT+SIA (which they call CDT+GT) gives KKT conditions. So we can restrict to CDT+GT points when searching for ex ante optimal policies.

One idea also is to relax to “list CDT+GT policies but without the inequalities and allowing complex numbers” which is now algebraic geometry territory (solving complex polynomials), and then filter for inequalities, real numbers, and global optimality afterwards.

From reading through the Sleeping Beauty literature, I learned about the Wichardt 2008 counterexample for multi-player ex ante optimality, which is relevant to UDT (and says why we shouldn’t expect nice things for UDT in general).

To give a brief summary, suppose Alice has 2 copies, who have the same source code and who can randomize independently. There are 2 coffee shops that the copies can decide to go to without communicating. They would really like (+5) to meet at the same coffee shop. Also, Bob hates Alice, his utility function is hers negated. Alice gets −1 utility (and Bob +1) if Bob goes to the coffee shop where both Alice-copies go. (Basically Alice is playing a “coordination” variant of absent-minded driver with herself, while also playing matching pennies with Bob)

The problem is that they can’t both select ex ante optimal policies in the Nash equilibrium sense. Alice needs to either 100% go to coffee shop A or B to be ex ante optimal; suppose it’s A. But then Bob needs to 100% go to coffee shop A to be ex ante optimal. Now Alice isn’t ex ante optimal anymore because it would have been more optimal for both her copies to go to coffee shop B.

If we treat it as a 3 player game then we have a Nash equilibrium (both Alices go to A, Bob goes to A). This is CDT+SIA optimal on everyone’s part (the SIA part doesn’t matter for these purposes). Since neither Alice-instance can unilaterally switch and expect things to (causally) work out well.

Given this impossibility result it’s not clear what UDT-like decision theories should do in these sorts of situations. Since it means not everyone can be ex ante optimal and maybe we need weaker conditions too.

[cousin_it]

I see, thanks! The Wichardt example is really interesting. I already kinda knew that UDT in multiplayer games doesn’t work too well (even in a simple 2-player asymmetric game with nonzero sum, equilibrium selection becomes a problem), but the nonexistence of Nash equilibria is even more fun. Would you say that CDT+SIA optimality works better than UDT for multiplayer games, and in how much generality?

[jessicata]

While it doesn’t globally “work better”, yes CDT+SIA is guaranteed to have solutions in multi player absent minded games, and UDT isn’t.

There are also EDT+FNC (aka EDT+GDH) equilibrium concepts, which do not always exist in multiplayer games (see “Imperfect-Recall Games: Equilibrium Concepts and Their Complexity”, Lemma 21).

I do think “multi player equilibria always exist” is a pretty desirable property for an equilibrium concept, although the lack of ex ante optimality is an issue. (Some of my previous writing on this: “Buridan’s ass in coordination games”, “Reducing collective rationality to individual optimization in common-payoff games using MCMC”, both making use of shared randomness, a workaround for Wichardt 2008 style counterexamples; as an offhand idea, it might be interesting to examine how logical inductor EDT handles such cases.)

Absent minded games can provide “easier” models of Newcomblike games; quoting a previous post,

Memoryless Cartesian environments can be used to define many familier decision problems (for example, the absent-minded driver problem, Newcomb’s problem with opaque or transparent boxes (assuming Omega runs a copy of the agent to make its prediction), counterfactual mugging (also assuming Omega simulates the agent)). Translating a decision problem to a memoryless Cartesian environment obviously requires making some Cartesian assumptions/​decisions, though; in the case of Newcomb’s problem, we have to isolate Omega’s simulation of the agent as a copy of the agent.

Which is part of why I’m focusing on them (as they seem less general /​ easier to analyze).

[cousin_it]

Yeah, agree on the point about absent-minded games, I think I realized it in 2014. But I didn’t make the jump to multiplayer absent-minded games. It’s cool that you explained it to me now, I’ll spend some time thinking it through.

Maybe the more general problem is I tend to abandon whole directions of thinking pretty easily if I see a “deep enough” problem with them. Abandoning probabilities because of absent-minded driver (as I mentioned in the toplevel comment); abandoning logical induction because of Diffractor’s result that the probability distribution LI converges on doesn’t itself satisfy the LI criterion and can be exploited by traders; abandoning most ideas on equilibrium selection because (edit) there’s just too many with no clear winner. Or maybe it’s a good heuristic and just sometimes works badly, idk.

[deleted]

[deleted]