I think the main reason why UDT is not discussed in academia is that it is not a sufficiently rigorous proposal, as well as there not being a published paper on it. Hilary Greaves says the following in this 80k episode:
Then as many of your listeners will know, in the space of AI research, people have been throwing around terms like ‘functional decision theory’ and ‘timeless decision theory’ and ‘updateless decision theory’. I think it’s a lot less clear exactly what these putative alternatives are supposed to be. The literature on those kinds of decision theories hasn’t been written up with the level of precision and rigor that characterizes the discussion of causal and evidential decision theory. So it’s a little bit unclear, at least to my likes, whether there’s genuinely a competitor to decision theory on the table there, or just some intriguing ideas that might one day in the future lead to a rigorous alternative.
I also think it is unclear to what extent UDT and updateless are different from existing ideas in academia that are prima facie similar, like McClennen’s (1990) resolute choice and Meacham’s (2010, §4.2) cohesive decision theory.[1] Resolute choice in particular has been discussed in a lot of detail, and for a long time (see the citations of McClennen’s book). (And, FWIW, my sense is that most philosophers think that resolute choice is irrational and/or doesn’t make sense, at least if it is cashed out as a decision rule based on preference.)
It also doesn’t help that it is unclear what the difference between FDT and UDT is supposed to be.
(If UDT is supposed to be an LDT of some sort, then you might want to check out Spohn’s (2012)[2] version of CDT, Fisher’s (n.d) disposition-based decision theory, and Poellinger’s (2013) discussion of Spohn’s theory, for ideas in academia that are similar to the LDT-part of the theory. And then there is also Schwarz’ critique of FDT, which would then also apply to UDT, at least partially.)
My own take, using the terminology listed here, is that the causalist version of Meacham’s cohesive decision theory is basically “updateless CDT”, that the evidentialist version is basically “updateless EDT”, and that a Spohn-CDT version of cohesive decision theory is basically “U(C)DT/F(C)DT”. I also think that resolute choice is much more permissive than e.g. cohesive decision theory and updatelessness. As a decision rule, it doesn’t recommend anything close to “maximizing EU relative to your prior”. Instead, it just states that (i) what you prefer ex ante in a dynamic choice problem should be the same as you what you prefer in the normalised version of the problem, and (ii) you should be dynamically consistent (i.e., the most preferred plan should not change throughout the decision problem).
It may be worth thinking about why proponents of a very popular idea in this community don’t know of its academic analogues, despite them having existed since the early 90s[1] and appearing on the introductory SEP page for dynamic choice.
Academics may in turn ask: clearly LessWrong has some blind spots, but how big?
It may be worth thinking about why proponents of a very popular idea in this community don’t know of its academic analogues
I don’t think this is fair, because even though component ideas behind UDT/FDT have academic analogues, it doesn’t look like someone put them together into a single decision theory formulation in academic literature, at least prior to MIRI’s “Cheating Death in Damascus” being published. Also “Cheating Death in Damascus” does cite both Meacham and Spohn (and others) and it seems excusable for me to have forgotten those references since they were both published after I wrote about UDT and again were only component ideas of it, plus I haven’t actively worked on decision theory for several years.
I think Sami’s comment is entirely fair given the language and framing of the original post. It is of course fine to forget about references, but e.g. “I find it curious that none of my ideas have a following in academia or have been reinvented/rediscovered by academia” and “Clearly academia has some blind spots, but how big?” reads like you don’t consider it a possilbity that you might have re-invented something yourself, and that academics are at fault for not taking up your ideas.
(It sucks to debate this, but ignoring it might be interpreted as tacit agreement. Maybe I should have considered the risk that something like this would happen and not written my OP.)
When I wrote the OP, I was pretty sure that the specific combination of ideas in UDT has not been invented or re-invented or have much of a following in academia, at least as of 2019 when Cheating Death in Damascus was published, because the authors of that paper obviously did a literature search and would have told me if they had found something very similar to UDT in the literature, and I think I also went through the papers it referenced as being related and did not find something that had all of the elements of UDT (that’s probably why your references look familiar to me). Plus FDT was apparently considered novel enough that the reviewers of the paper didn’t tell the authors that they had to call it by the name of an existing academic decision theory.
So it’s not that I “don’t consider it a possibility that you might have re-invented something yourself” but that I had good reason to think that’s not the case?
I think there is nothing surprising that small community of nerds writing in spare time has blind spots, but when large professional community has such blind spots that’s surprising.
On your first point: as Sami writes, resolute choice is mentioned in the introductory SEP article on dynamic choice (it even has its own section!), as well as in the SEP article on decision theory. And SEP is the first place you go when you want to learn about philosophical topics and find references.
On your second point: as I wrote in my comment above, (i) academics have produced seemingly similar ideas to e.g. updatelessness (well before they were written up on LW) so it is unclear why academics should engage with less rigorous, unpublished proposals that appear to be similar (in other words, I don’t think the phrase “blind spots” is warranted), and (ii) when academics have commented on or engaged with LW DT ideas, they have to my knowledge largely been critical (e.g. see the post by Wolfgang Schwarz I linked above, as well as the quote from Greaves)[1].
To clarify, by “blind spot” I wasn’t complaining that academia isn’t engaging specifically with posts written up on LW, but more that nobody in academia seems to think that the combination of “updateless+logical” is clearly the most important or promising direction to explore in decision theory.
I think the main reason why UDT is not discussed in academia is that it is not a sufficiently rigorous proposal, as well as there not being a published paper on it.
The reason for the former is that I (and others) have been unable to find a rigorous formulation of it that doesn’t have serious open problems. (I and I guess other decision theory researchers in this community currently think that UDT is more of a relatively promising direction to explore, rather than a good decision theory per se.)
And the reason for the latter is the above, plus my personal distaste for writing/publishing academic papers (which I talked about elsewhere in this thread), plus FDT having been published which seems close enough to me.
Thank for the references in the rest of your comment. I think I’ve come across Meacham 2010 and Spohn 2012 before, but forgot about them as I haven’t been working actively on decision theory for a while. It does seem that Meacham’s cohesive decision theory is equivalent to updateless EDT/CDT. (BTW in The Absent-Minded Driver I referenced a 1997 paper that also has an idea similar to updatelessness, although the authors didn’t like it.)
On a quick skim of Spohn 2012 I didn’t see something that looks like LDT or “algorithmic/logical agent ontology” but it’s quite long/dense so I’ll take your word on it for now. Still, it seems like none of the academic papers put all of the pieces together in a single decision theory proposal that’s equivalent to UDT or FDT?
(Please note that UDT as originally described was actually updateless/evidential/logical, not causalist as you wrote in the post that you linked. This has been a historical disagreement between me and Eliezer, where in I leaned towards evidential and he leans towards causal, although these days I just say that I’m confused and don’t know what to think.)
The reason for the former is that I (and others) have been unable to find a rigorous formulation of it that doesn’t have serious open problems. (I and I guess other decision theory researchers in this community currently think that UDT is more of a relatively promising direction to explore, rather than a good decision theory per se.)
That’s fair. But what is it then that you expect academics to engage with? How would you describe this research direction, and why do you think it’s interesting and/or important?
To quickly recap the history, people on LW noticed some clear issues with “updating” and “physicalist ontology” of the most popular decision theories at the time (CDT/EDT), and thought that switching to “updatelessness” and “logical/algorithmic ontology” was an obvious improvement. (I was the first person to put the two pieces together in an explicit formulation, but they were already being talked about / hinted at in the community.) Initially people were really excited because the resulting decision theories (UDT/FDT) seemed to solve a lot of open problems in one swoop, but then pretty quickly and over time we noticed more and more problems with UDT/FDT that seem to have no clear fixes.
So we were initially excited but then increasingly puzzled/confused, and I guess I was expecting at least some academics to follow a similar path, either through engagement with LW ideas (why should they be bothered that much by lack of academic publication?), or from independent invention. Instead academia seems to still be in a state similar to LW when I posted UDT, i.e., the ideas are floating in the air separately and nobody has put them together yet? (Or I guess that was the state of academia before FDT was published in an academic journal, so now the situation is more like some outsiders put the pieces together in a formal publication, but still no academic is following a similar path as us.)
I guess it’s also possible that academia sort of foresaw or knew all the problems that we’d eventually find with UDT/FDT and that’s why they didn’t get excited in the first place. I haven’t looked into academic DT literature in years, so you’re probably more familiar with it. Do you know if they’re puzzled/confused by the same problems that we are? Or what are they mostly working on / arguing about these days?
There are many many interesting questions in decision theory, and “dimensions” along which decision theories can vary, not just the three usually discussed on LessWrong. It’s not clear to me why (i) philosophers should focus on the dimensions you primarily seem to be interested in, and (ii) what is so special about the particular combination you mention (is there some interesting interaction I don’t know about maybe?). Furthermore, note that most philosophers probably do not share your intuitions: I’m pretty sure most of them would e.g. pay in counterfactual mugging. (And I have not seen a good case for why it would be rational to pay.) I don’t mean to be snarky, but you could just be wrong about what the open problems are.
I haven’t looked into academic DT literature in years, so you’re probably more familiar with it. Do you know if they’re puzzled/confused by the same problems that we are?
I wouldn’t say so, no. But I’m not entirely sure if I understand what the open problems are. Reading your list of seven issues, I either (i) don’t understand what you are asking, (ii) disagree with the framing/think the question is misguided, or (iii) think there is an obvious answer (which makes me think that I’m missing something). With that said, I haven’t read all the posts you reference, so perhaps I should read those first.
There are many many interesting questions in decision theory, and “dimensions” along which decision theories can vary, not just the three usually discussed on LessWrong.
It would be interesting to get an overview of what these are. Or if that’s too hard to write down, and there are no ready references, what are your own interests in decision theory?
what is so special about the particular combination you mention
Furthermore, note that most philosophers probably do not share your intuitions
Agreed, but my intuitions don’t seem so unpopular outside academia or so obviously wrong that there should be so few academic philosophers who do share them.
I’m pretty sure most of them would e.g. pay in counterfactual mugging. (And I have not seen a good case for why it would be rational to pay.)
I’m not sure I wouldn’t pay either. I see it as more of an interesting puzzle than having a definitive answer. ETA: Although I’m more certain that we should build AIs that do pay. Is that also unclear to you? (If so why might we not want to build such AIs?)
I don’t mean to be snarky, but you could just be wrong about what the open problems are.
Yeah, I’m trying to keep an open mind about that. :)
With that said, I haven’t read all the posts you reference, so perhaps I should read those first.
Cool, I’d be interested in any further feedback when you’re ready to give them.
It would be interesting to get an overview of what these are. Or if that’s too hard to write down, and there are no ready references, what are your own interests in decision theory?
I’m not sure I wouldn’t pay either. I see it as more of an interesting puzzle than having a definitive answer. ETA: Although I’m more certain that we should build AIs that do pay. Is that also unclear to you? (If so why might we not want to build such AIs?)
Okay, interesting! I thought UDT was meant to pay in CM, and that you were convinced of (some version of) UDT.
On the point about AI (not directly responding to your question, to which I don’t have an answer): I think it’s really important to be clear about whether we are discussing normative, constructive or descriptive decision theory (using Elliott Thornley’s distinction here). For example, the answers to “is updatelessness normatively compelling?”, “should we build an updateless AI?” and “will some agents (e.g. advanced AIs) commit to being updateless?” will most likely come apart (it seems to me). And I think that discussions on LW about decision theory are often muddled due to not making clear what is being discussed.
(BTW this issue/doubt about whether UDT / paying CM is normative for humans is item 1 in the above linked post. Thought I’d point that out since it may not be obvious at first glance.)
And I think that discussions on LW about decision theory are often muddled due to not making clear what is being discussed.
Yeah I agree with this to some extent, and try to point out such confusions or make such distinctions when appropriate. (Such as in the CM / indexical values case.) Do you have more examples where making such distinctions would be helpful?
I wrote “I’m really not sure at this point whether UDT is even on the right track” in UDT shows that decision theory is more puzzling than ever which I think you’ve read? Did you perhaps miss that part?
Yes, missed or forgot about that sentence, sorry.
(BTW this issue/doubt about whether UDT / paying CM is normative for humans is item 1 in the above linked post. Thought I’d point that out since it may not be obvious at first glance.)
Thanks.
Do you have more examples where making such distinctions would be helpful?
I was mostly thinking about discussions surrounding what the “correct” decision theory, is whether you should pay in CM, and so on.
Thanks Sylvester! Yep it looks like cohesive decision theory is basically original UDT. Do you know what the state of the art is in terms of philosophical critiques of cohesive decision theory? Any good ones? Any good responses to the critiques?
Cohesive decision theory lacks the logical/algorithmic ontology of UDT and is closer to what we call “updateless EDT/CDT” (the paper itself talks about cohesive versions of both).
Also interested in a response from Sylvester, but I would guess that one of the main critiques is something like Will MacAskill’s Bomb thought experiment, or just intuitions for paying the counterfactual mugger. From my perspective, these do have a point when it comes to humans, since humans seemingly have indexical values, and one way to explain why UDT makes recommendations in these thought experiments that look “bizarre” to many humans is that it assumes away indexical values (via the type signature of its utility function). (It was an implicit and not totally intentional assumption, but it’s unclear how to remove the assumption while retaining nice properties associated with updatelessness.) I’m unsure if indexical values themselves are normative or philosophically justified, and they are probably irrelevant or undesirable when it comes to AIs, but I guess academic philosophers probably take them more for granted and are not as interested in AI (and therefore take a dimmer view on updatelessness/cohesiveness).
But yeah, if there are good critiques/responses aside from these, it would be interesting to learn them.
I don’t think cohesive decision theory is being discussed much, but I’m not sure. Perhaps because the theory is mainly used to argue against the claim that “every decision rule will lead agents who can’t bind themselves to disaster” (p. 20, footnote 34) in the paper, and discussion of its independent interest is relegated to a footnote (footnote 34).
I think the main reason why UDT is not discussed in academia is that it is not a sufficiently rigorous proposal, as well as there not being a published paper on it. Hilary Greaves says the following in this 80k episode:
I also think it is unclear to what extent UDT and updateless are different from existing ideas in academia that are prima facie similar, like McClennen’s (1990) resolute choice and Meacham’s (2010, §4.2) cohesive decision theory.[1] Resolute choice in particular has been discussed in a lot of detail, and for a long time (see the citations of McClennen’s book). (And, FWIW, my sense is that most philosophers think that resolute choice is irrational and/or doesn’t make sense, at least if it is cashed out as a decision rule based on preference.)
It also doesn’t help that it is unclear what the difference between FDT and UDT is supposed to be.
(If UDT is supposed to be an LDT of some sort, then you might want to check out Spohn’s (2012)[2] version of CDT, Fisher’s (n.d) disposition-based decision theory, and Poellinger’s (2013) discussion of Spohn’s theory, for ideas in academia that are similar to the LDT-part of the theory. And then there is also Schwarz’ critique of FDT, which would then also apply to UDT, at least partially.)
My own take, using the terminology listed here, is that the causalist version of Meacham’s cohesive decision theory is basically “updateless CDT”, that the evidentialist version is basically “updateless EDT”, and that a Spohn-CDT version of cohesive decision theory is basically “U(C)DT/F(C)DT”. I also think that resolute choice is much more permissive than e.g. cohesive decision theory and updatelessness. As a decision rule, it doesn’t recommend anything close to “maximizing EU relative to your prior”. Instead, it just states that (i) what you prefer ex ante in a dynamic choice problem should be the same as you what you prefer in the normalised version of the problem, and (ii) you should be dynamically consistent (i.e., the most preferred plan should not change throughout the decision problem).
Note that in the published article, it says that the article was received in 2008.
It may be worth thinking about why proponents of a very popular idea in this community don’t know of its academic analogues, despite them having existed since the early 90s[1] and appearing on the introductory SEP page for dynamic choice.
Academics may in turn ask: clearly LessWrong has some blind spots, but how big?
And it’s not like these have been forgotton; e.g., McClennen’s (1990) work still gets cited regularly.
I don’t think this is fair, because even though component ideas behind UDT/FDT have academic analogues, it doesn’t look like someone put them together into a single decision theory formulation in academic literature, at least prior to MIRI’s “Cheating Death in Damascus” being published. Also “Cheating Death in Damascus” does cite both Meacham and Spohn (and others) and it seems excusable for me to have forgotten those references since they were both published after I wrote about UDT and again were only component ideas of it, plus I haven’t actively worked on decision theory for several years.
I think Sami’s comment is entirely fair given the language and framing of the original post. It is of course fine to forget about references, but e.g. “I find it curious that none of my ideas have a following in academia or have been reinvented/rediscovered by academia” and “Clearly academia has some blind spots, but how big?” reads like you don’t consider it a possilbity that you might have re-invented something yourself, and that academics are at fault for not taking up your ideas.
(It sucks to debate this, but ignoring it might be interpreted as tacit agreement. Maybe I should have considered the risk that something like this would happen and not written my OP.)
When I wrote the OP, I was pretty sure that the specific combination of ideas in UDT has not been invented or re-invented or have much of a following in academia, at least as of 2019 when Cheating Death in Damascus was published, because the authors of that paper obviously did a literature search and would have told me if they had found something very similar to UDT in the literature, and I think I also went through the papers it referenced as being related and did not find something that had all of the elements of UDT (that’s probably why your references look familiar to me). Plus FDT was apparently considered novel enough that the reviewers of the paper didn’t tell the authors that they had to call it by the name of an existing academic decision theory.
So it’s not that I “don’t consider it a possibility that you might have re-invented something yourself” but that I had good reason to think that’s not the case?
I think there is nothing surprising that small community of nerds writing in spare time has blind spots, but when large professional community has such blind spots that’s surprising.
On your first point: as Sami writes, resolute choice is mentioned in the introductory SEP article on dynamic choice (it even has its own section!), as well as in the SEP article on decision theory. And SEP is the first place you go when you want to learn about philosophical topics and find references.
On your second point: as I wrote in my comment above, (i) academics have produced seemingly similar ideas to e.g. updatelessness (well before they were written up on LW) so it is unclear why academics should engage with less rigorous, unpublished proposals that appear to be similar (in other words, I don’t think the phrase “blind spots” is warranted), and (ii) when academics have commented on or engaged with LW DT ideas, they have to my knowledge largely been critical (e.g. see the post by Wolfgang Schwarz I linked above, as well as the quote from Greaves)[1].
Cheating Death in Damascus getting published in the Journal of Philosophy is a notable exception though!
To clarify, by “blind spot” I wasn’t complaining that academia isn’t engaging specifically with posts written up on LW, but more that nobody in academia seems to think that the combination of “updateless+logical” is clearly the most important or promising direction to explore in decision theory.
The reason for the former is that I (and others) have been unable to find a rigorous formulation of it that doesn’t have serious open problems. (I and I guess other decision theory researchers in this community currently think that UDT is more of a relatively promising direction to explore, rather than a good decision theory per se.)
And the reason for the latter is the above, plus my personal distaste for writing/publishing academic papers (which I talked about elsewhere in this thread), plus FDT having been published which seems close enough to me.
Thank for the references in the rest of your comment. I think I’ve come across Meacham 2010 and Spohn 2012 before, but forgot about them as I haven’t been working actively on decision theory for a while. It does seem that Meacham’s cohesive decision theory is equivalent to updateless EDT/CDT. (BTW in The Absent-Minded Driver I referenced a 1997 paper that also has an idea similar to updatelessness, although the authors didn’t like it.)
On a quick skim of Spohn 2012 I didn’t see something that looks like LDT or “algorithmic/logical agent ontology” but it’s quite long/dense so I’ll take your word on it for now. Still, it seems like none of the academic papers put all of the pieces together in a single decision theory proposal that’s equivalent to UDT or FDT?
(Please note that UDT as originally described was actually updateless/evidential/logical, not causalist as you wrote in the post that you linked. This has been a historical disagreement between me and Eliezer, where in I leaned towards evidential and he leans towards causal, although these days I just say that I’m confused and don’t know what to think.)
That’s fair. But what is it then that you expect academics to engage with? How would you describe this research direction, and why do you think it’s interesting and/or important?
To quickly recap the history, people on LW noticed some clear issues with “updating” and “physicalist ontology” of the most popular decision theories at the time (CDT/EDT), and thought that switching to “updatelessness” and “logical/algorithmic ontology” was an obvious improvement. (I was the first person to put the two pieces together in an explicit formulation, but they were already being talked about / hinted at in the community.) Initially people were really excited because the resulting decision theories (UDT/FDT) seemed to solve a lot of open problems in one swoop, but then pretty quickly and over time we noticed more and more problems with UDT/FDT that seem to have no clear fixes.
So we were initially excited but then increasingly puzzled/confused, and I guess I was expecting at least some academics to follow a similar path, either through engagement with LW ideas (why should they be bothered that much by lack of academic publication?), or from independent invention. Instead academia seems to still be in a state similar to LW when I posted UDT, i.e., the ideas are floating in the air separately and nobody has put them together yet? (Or I guess that was the state of academia before FDT was published in an academic journal, so now the situation is more like some outsiders put the pieces together in a formal publication, but still no academic is following a similar path as us.)
I guess it’s also possible that academia sort of foresaw or knew all the problems that we’d eventually find with UDT/FDT and that’s why they didn’t get excited in the first place. I haven’t looked into academic DT literature in years, so you’re probably more familiar with it. Do you know if they’re puzzled/confused by the same problems that we are? Or what are they mostly working on / arguing about these days?
There are many many interesting questions in decision theory, and “dimensions” along which decision theories can vary, not just the three usually discussed on LessWrong. It’s not clear to me why (i) philosophers should focus on the dimensions you primarily seem to be interested in, and (ii) what is so special about the particular combination you mention (is there some interesting interaction I don’t know about maybe?). Furthermore, note that most philosophers probably do not share your intuitions: I’m pretty sure most of them would e.g. pay in counterfactual mugging. (And I have not seen a good case for why it would be rational to pay.) I don’t mean to be snarky, but you could just be wrong about what the open problems are.
I wouldn’t say so, no. But I’m not entirely sure if I understand what the open problems are. Reading your list of seven issues, I either (i) don’t understand what you are asking, (ii) disagree with the framing/think the question is misguided, or (iii) think there is an obvious answer (which makes me think that I’m missing something). With that said, I haven’t read all the posts you reference, so perhaps I should read those first.
It would be interesting to get an overview of what these are. Or if that’s too hard to write down, and there are no ready references, what are your own interests in decision theory?
As I mentioned in the previous comment, it happens to solve (or at least seemed like a good step towards solving) a lot of problems I was interested in at the time.
Agreed, but my intuitions don’t seem so unpopular outside academia or so obviously wrong that there should be so few academic philosophers who do share them.
I’m not sure I wouldn’t pay either. I see it as more of an interesting puzzle than having a definitive answer. ETA: Although I’m more certain that we should build AIs that do pay. Is that also unclear to you? (If so why might we not want to build such AIs?)
Yeah, I’m trying to keep an open mind about that. :)
Cool, I’d be interested in any further feedback when you’re ready to give them.
Yeah, that would be too hard. You might want to look at these SEP entries: Decision Theory, Normative Theories of Rational Choice: Expected Utility, Normative Theories of Rational Choice: Rivals to Expected Utility and Causal Decision Theory. To give an example of what I’m interested in, I think it is really important to take into account unawareness and awareness growth (see §5.3 of the first entry listed above) when thinking about how ordinary agents should make decisions. (Also see this post.)
Okay, interesting! I thought UDT was meant to pay in CM, and that you were convinced of (some version of) UDT.
On the point about AI (not directly responding to your question, to which I don’t have an answer): I think it’s really important to be clear about whether we are discussing normative, constructive or descriptive decision theory (using Elliott Thornley’s distinction here). For example, the answers to “is updatelessness normatively compelling?”, “should we build an updateless AI?” and “will some agents (e.g. advanced AIs) commit to being updateless?” will most likely come apart (it seems to me). And I think that discussions on LW about decision theory are often muddled due to not making clear what is being discussed.
Thanks, will look into your references.
I wrote “I’m really not sure at this point whether UDT is even on the right track” in UDT shows that decision theory is more puzzling than ever which I think you’ve read? Did you perhaps miss that part?
(BTW this issue/doubt about whether UDT / paying CM is normative for humans is item 1 in the above linked post. Thought I’d point that out since it may not be obvious at first glance.)
Yeah I agree with this to some extent, and try to point out such confusions or make such distinctions when appropriate. (Such as in the CM / indexical values case.) Do you have more examples where making such distinctions would be helpful?
Yes, missed or forgot about that sentence, sorry.
Thanks.
I was mostly thinking about discussions surrounding what the “correct” decision theory, is whether you should pay in CM, and so on.
Thanks Sylvester! Yep it looks like cohesive decision theory is basically original UDT. Do you know what the state of the art is in terms of philosophical critiques of cohesive decision theory? Any good ones? Any good responses to the critiques?
Cohesive decision theory lacks the logical/algorithmic ontology of UDT and is closer to what we call “updateless EDT/CDT” (the paper itself talks about cohesive versions of both).
Also interested in a response from Sylvester, but I would guess that one of the main critiques is something like Will MacAskill’s Bomb thought experiment, or just intuitions for paying the counterfactual mugger. From my perspective, these do have a point when it comes to humans, since humans seemingly have indexical values, and one way to explain why UDT makes recommendations in these thought experiments that look “bizarre” to many humans is that it assumes away indexical values (via the type signature of its utility function). (It was an implicit and not totally intentional assumption, but it’s unclear how to remove the assumption while retaining nice properties associated with updatelessness.) I’m unsure if indexical values themselves are normative or philosophically justified, and they are probably irrelevant or undesirable when it comes to AIs, but I guess academic philosophers probably take them more for granted and are not as interested in AI (and therefore take a dimmer view on updatelessness/cohesiveness).
But yeah, if there are good critiques/responses aside from these, it would be interesting to learn them.
I don’t think cohesive decision theory is being discussed much, but I’m not sure. Perhaps because the theory is mainly used to argue against the claim that “every decision rule will lead agents who can’t bind themselves to disaster” (p. 20, footnote 34) in the paper, and discussion of its independent interest is relegated to a footnote (footnote 34).
OK, thanks. So then the mystery remains why academic philosophy isn’t more interested in this.