and Silvia’s work
Typo: it’s Sylvia.
Sylvester Kollin
Karma: 340
I wrote “I’m really not sure at this point whether UDT is even on the right track” in UDT shows that decision theory is more puzzling than ever which I think you’ve read? Did you perhaps miss that part?
Yes, missed or forgot about that sentence, sorry.
(BTW this issue/doubt about whether UDT / paying CM is normative for humans is item 1 in the above linked post. Thought I’d point that out since it may not be obvious at first glance.)
Thanks.
Do you have more examples where making such distinctions would be helpful?
I was mostly thinking about discussions surrounding what the “correct” decision theory, is whether you should pay in CM, and so on.
Here’s a related idea that is maybe clearer: Suppose an agent has the ability to self-modify to use any decision theory, would they decide to stick with their current decision theory? (I’m actually not sure what term has been explicitly defined to mean this, so I’ll just call it “self-endorsement” for now.)
This sounds similar to what’s called “self-recommendation”—see e.g. Skyrms (1982, pp. 707-709), Meacham (2010, §3.3) and Pettigrew (2023). In the abstract Pettigrew writes: “A decision theory is self-recommending if, when you ask it which decision theory you should use, it considers itself to be among the permissible options.”.
I have actually been thinking about ways of extending Pettigrew’s work to theories of dynamic choice. That is: is sophistication/resoluteness self-recommending? I don’t think it is immediately clear what the answers are, and it might depend on the interpretations of sophistication and resoluteness one adopts, but yeah, I do agree that it seems like sophistication might be self-undermining.
Thanks for the clarification!
I do understand from the SEP, like Wei, that sophisticated means “backwards planning”, and resolute means “being able to commit to a policy” (correct me if I’m wrong).
That seems roughly correct, but note that there are different interpretations of resolute choice floating around[1], and I think McClennen’s (1990) presentation is somewhat unclear at times. Sometimes resoluteness seems to be about the ability to make internal commitments, and other times it seems to be about being sensitive to the dynamic context in a particular way, and I think these can come apart. You might be interested in these notes I wrote while reading McClennen’s book.
My usage of “dynamic instability” (which might be contrary to academic usage) was indeed what Wei mentions: “not needing commitments”. Or equivalently, I say a decision theory is dynamically stable if itself and its resolute version always act the same.
Then that sounds a bit question-begging. Do you think dynamic instability is a problem (normatively speaking)?
- ^
See e.g. Gauthier (1997) and Buchak (2013, §6).
- ^
I think Sami’s comment is entirely fair given the language and framing of the original post. It is of course fine to forget about references, but e.g. “I find it curious that none of my ideas have a following in academia or have been reinvented/rediscovered by academia” and “Clearly academia has some blind spots, but how big?” reads like you don’t consider it a possilbity that you might have re-invented something yourself, and that academics are at fault for not taking up your ideas.
I don’t think cohesive decision theory is being discussed much, but I’m not sure. Perhaps because the theory is mainly used to argue against the claim that “every decision rule will lead agents who can’t bind themselves to disaster” (p. 20, footnote 34) in the paper, and discussion of its independent interest is relegated to a footnote (footnote 34).
It would be interesting to get an overview of what these are. Or if that’s too hard to write down, and there are no ready references, what are your own interests in decision theory?
Yeah, that would be too hard. You might want to look at these SEP entries: Decision Theory, Normative Theories of Rational Choice: Expected Utility, Normative Theories of Rational Choice: Rivals to Expected Utility and Causal Decision Theory. To give an example of what I’m interested in, I think it is really important to take into account unawareness and awareness growth (see §5.3 of the first entry listed above) when thinking about how ordinary agents should make decisions. (Also see this post.)
I’m not sure I wouldn’t pay either. I see it as more of an interesting puzzle than having a definitive answer. ETA: Although I’m more certain that we should build AIs that do pay. Is that also unclear to you? (If so why might we not want to build such AIs?)
Okay, interesting! I thought UDT was meant to pay in CM, and that you were convinced of (some version of) UDT.
On the point about AI (not directly responding to your question, to which I don’t have an answer): I think it’s really important to be clear about whether we are discussing normative, constructive or descriptive decision theory (using Elliott Thornley’s distinction here). For example, the answers to “is updatelessness normatively compelling?”, “should we build an updateless AI?” and “will some agents (e.g. advanced AIs) commit to being updateless?” will most likely come apart (it seems to me). And I think that discussions on LW about decision theory are often muddled due to not making clear what is being discussed.
There are many many interesting questions in decision theory, and “dimensions” along which decision theories can vary, not just the three usually discussed on LessWrong. It’s not clear to me why (i) philosophers should focus on the dimensions you primarily seem to be interested in, and (ii) what is so special about the particular combination you mention (is there some interesting interaction I don’t know about maybe?). Furthermore, note that most philosophers probably do not share your intuitions: I’m pretty sure most of them would e.g. pay in counterfactual mugging. (And I have not seen a good case for why it would be rational to pay.) I don’t mean to be snarky, but you could just be wrong about what the open problems are.
I haven’t looked into academic DT literature in years, so you’re probably more familiar with it. Do you know if they’re puzzled/confused by the same problems that we are?
I wouldn’t say so, no. But I’m not entirely sure if I understand what the open problems are. Reading your list of seven issues, I either (i) don’t understand what you are asking, (ii) disagree with the framing/think the question is misguided, or (iii) think there is an obvious answer (which makes me think that I’m missing something). With that said, I haven’t read all the posts you reference, so perhaps I should read those first.
This is indeed what happens to the best-known decision theories (CDT and EDT): they want to commit to paying, but if they don’t, by the time they get to the Heads world they don’t pay. We call this dynamic instability, because different (temporal) versions of the agent seem to be working against each other.
Unless you are using “dynamic stability” to mean something other than “dynamic consistency”, I don’t think this is quite right. The standard philosophical theory of dynamic choice, sophisticated choice (see e.g. the SEP entry on decision theory), would not pay but is still dynamically consistent.
The reason for the former is that I (and others) have been unable to find a rigorous formulation of it that doesn’t have serious open problems. (I and I guess other decision theory researchers in this community currently think that UDT is more of a relatively promising direction to explore, rather than a good decision theory per se.)
That’s fair. But what is it then that you expect academics to engage with? How would you describe this research direction, and why do you think it’s interesting and/or important?
Could you perhaps say something about what a Kripkean semantics would look like for your logic?
On your first point: as Sami writes, resolute choice is mentioned in the introductory SEP article on dynamic choice (it even has its own section!), as well as in the SEP article on decision theory. And SEP is the first place you go when you want to learn about philosophical topics and find references.
On your second point: as I wrote in my comment above, (i) academics have produced seemingly similar ideas to e.g. updatelessness (well before they were written up on LW) so it is unclear why academics should engage with less rigorous, unpublished proposals that appear to be similar (in other words, I don’t think the phrase “blind spots” is warranted), and (ii) when academics have commented on or engaged with LW DT ideas, they have to my knowledge largely been critical (e.g. see the post by Wolfgang Schwarz I linked above, as well as the quote from Greaves)[1].
- ^
Cheating Death in Damascus getting published in the Journal of Philosophy is a notable exception though!
- ^
I think the main reason why UDT is not discussed in academia is that it is not a sufficiently rigorous proposal, as well as there not being a published paper on it. Hilary Greaves says the following in this 80k episode:
Then as many of your listeners will know, in the space of AI research, people have been throwing around terms like ‘functional decision theory’ and ‘timeless decision theory’ and ‘updateless decision theory’. I think it’s a lot less clear exactly what these putative alternatives are supposed to be. The literature on those kinds of decision theories hasn’t been written up with the level of precision and rigor that characterizes the discussion of causal and evidential decision theory. So it’s a little bit unclear, at least to my likes, whether there’s genuinely a competitor to decision theory on the table there, or just some intriguing ideas that might one day in the future lead to a rigorous alternative.
I also think it is unclear to what extent UDT and updateless are different from existing ideas in academia that are prima facie similar, like McClennen’s (1990) resolute choice and Meacham’s (2010, §4.2) cohesive decision theory.[1] Resolute choice in particular has been discussed in a lot of detail, and for a long time (see the citations of McClennen’s book). (And, FWIW, my sense is that most philosophers think that resolute choice is irrational and/or doesn’t make sense, at least if it is cashed out as a choice rule.)
It also doesn’t help that it is unclear what the difference between FDT and UDT is supposed to be.
(If UDT is supposed to be an LDT of some sort, then you might want to check out Spohn’s (2012)[2] version of CDT, Fisher’s (n.d) disposition-based decision theory, and Poellinger’s (2013) discussion of Spohn’s theory, for ideas in academia that are similar to the LDT-part of the theory. And then there is also Schwarz’ critique of FDT, which would then also apply to UDT, at least partially.)- ^
My own take, using the terminology listed here, is that the causalist version of Meacham’s cohesive decision theory is basically “updateless CDT”, that the evidentialist version is basically “updateless EDT”, and that a Spohn-CDT version of cohesive decision theory is basically “U(C)DT/F(C)DT”. I also think that resolute choice is much more permissive than e.g. cohesive decision theory and updatelessness. As a choice rule, it doesn’t recommend anything close to “maximizing EU relative to your prior”. Instead, it just states that (i) how you act ex ante in a dynamic choice problem should be the same as you would act in the normalised version of the problem, and (ii) you should be dynamically consistent (i.e., the most preferred plan should not change throughout the decision problem).
- ^
Note that in the published article, it says that the article was received in 2008.
- ^
Thanks.
I am pretty sure they’re interchangeable however.
Do you have a reference for this? Or perhaps there is a quick proof that could convince me?
You might also find the following cases interesting (with self-locating uncertainty as an additional dimension), from this post.
Sleeping Newcomb-1. Some researchers, led by the infamous superintelligence Omega, are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the toss of a biased coin (Heads: once; Tails: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. The weight of the coin is determined by what the superintelligence predicts that you would say when you are awakened and asked to what degree ought you believe that the outcome of the coin toss is Heads. Specifically, if the superintelligence predicted that you would have a degree of belief in Heads, then they will have weighted the coin such that the ‘objective chance’ of Heads is . So, when you are awakened, to what degree ought you believe that the outcome of the coin toss is Heads?
Sleeping Newcomb-2. Some researchers, led by the superintelligence Omega, are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the toss of a biased coin (Heads: once; Tails: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. The weight of the coin is determined by what the superintelligence predicts your response would be when you are awakened and asked to what degree you ought to believe that the outcome of the coin toss is Heads. Specifically, if Omega predicted that you would have a degree of belief in Heads, then they will have weighted the coin such that the ‘objective chance’ of Heads is . Then: when you are in fact awakened, to what degree ought you believe that the outcome of the coin toss is Heads?
Epistemic Constraint: The probability distribution which the agent settles on cannot be self-refuting according to the beliefs. It must be a fixed point of : a such that .
Minor: there might be cases in which there is a fixed point , but where the agent doesn’t literally converge or deliberate their way to it, right? (Because you are only looking for to satisfy the conditions of Brouwer/Kakutani, and not, say, Banach, right?) In other words, it might not always be accurate to say that the agent “settles on ”. EDIT: oh, maybe you are just using “settles on” in the colloquial way.
A common trope is for magic to work only when you believe in it. For example, in Harry Potter, you can only get to the magical train platform 9 3⁄4 if you believe that you can pass through the wall to get there.
Are you familiar with Greaves’ (2013) epistemic decision theory? These types of cases are precisely the ones she considers, although she is entirely focused on the epistemic side of things. For example (p. 916):
Leap. Bob stands on the brink of a chasm, summoning up the courage to try and leap across it. Confidence helps him in such situations: specifically, for any value of between and , if Bob attempted to leap across the chasm while having degree of belief that he would succeed, his chance of success would then be . What credence in success is it epistemically rational for Bob to have?
And even more interesting cases (p. 917):
Embezzlement. One of Charlie’s colleagues is accused of embezzling funds. Charlie happens to have conclusive evidence that her colleague is guilty. She is to be interviewed by the disciplinary tribunal. But Charlie’s colleague has had an opportunity to randomize the content of several otherwise informative files (files, let us say, that the tribunal will want to examine if Charlie gives a damning testimony). Further, in so far as the colleague thinks that Charlie believes him guilty, he will have done so. Specifically, if is the colleague’s prediction for Charlie’s degree of belief that he’s guilty, then there is a chance that he has set in motion a process by which each proposition originally in the files is replaced by its own negation if a fair coin lands Heads, and is left unaltered if the coin lands Tails. The colleague is a very reliable predictor of Charlie’s doxastic states. After such randomization (if any occurred), Charlie has now read the files; they (now) purport to testify to the truth of propositions . Charlie’s credence in each of the propositions conditional on the proposition that the files have been randomized, is ; her credence in each conditional on the proposition that the files have not been randomized is . What credence is it epistemically rational for Charlie to have in the proposition that her colleague is guilty and in the propositions that the files purport to testify to the truth of?
In particular, Greaves’ (2013, §8, pp. 43-49) epistemic version of Arntzenius’ (2008) deliberational (causal) decision theory might be seen as a way of making sense of the first part of your theory. The idea, inspired by Skyrms (1990), is that deciding on a credence involves a cycle of calculating epistemic expected utility (measured by a proper scoring rule), adjusting credences, and recalculating utilities until an equilibrium is
obtained. For example, in Leap above, epistemic D(C)DT would find any credence permissible. And I guess that the second part of your theory serves as a way of breaking ties.
We assume two rules of inference:
Necessitation:
Distributivity:Is there a reason why this differs from the standard presentation of K? Normally, I think you would say that K is generated by the following (coupled with substitution):
Axioms:
- All tautologies of propositional logic.
- Distribution: .Rules of inference:
- Necessitation: .
- Modus ponens: .
And, second, the agent will continually implement that plan, even if this makes it locally choose counter-preferentially at some future node.
Nitpick: IIRC, McClennen never talks about counter-preferential choice. Rather, that’s Gauthier’s (1997) approach to resoluteness.
as devised by Bryan Skyrms and Gerald Rothfus (cf Rothfus 2020b).
Found a typo: it is supposed to be Gerard. (It is also misspelt in the reference list.)
Nitpicks: (1) the vNM theorem is about preference, not choice and behavior; and (2) “can’t be money pumped” is not one of the conditions in the theorem.