# De­ci­sion Theory

TagLast edit: 19 Mar 2023 21:34 UTC by

Decision theory is the study of principles and algorithms for making correct decisions—that is, decisions that allow an agent to achieve better outcomes with respect to its goals. Every action at least implicitly represents a decision under uncertainty: in a state of partial knowledge, something has to be done, even if that something turns out to be nothing (call it “the null action”). Even if you don’t know how you make decisions, decisions do get made, and so there has to be some underlying mechanism. What is it? And how can it be done better? Decision theory has the answers.

Note: this page needs to be updated with content regarding Functional Decision Theory, the latest theory from MIRI.

A core idea in decision theory is that of expected utility maximization, usually intractable to directly calculate in practice, but an invaluable theoretical concept. An agent assigns utility to every possible outcome: a real number representing the goodness or desirability of that outcome. The mapping of outcomes to utilities is called the agent’s utility function. (The utility function is said to be invariant under affine transformations: that is, the utilities can be scaled or translated by a constant while resulting in all the same decisions.) For every action that the agent could take, sum over the utilities of the various possible outcomes weighted by their probability: this is the expected utility of the action, and the action with the highest expected utility is to be chosen.

## Thought experiments

The limitations and pathologies of decision theories can be analyzed by considering the decisions they suggest in the certain idealized situations that stretch the limits of decision theory’s applicability. Some of the thought experiments more frequently discussed on LW include:

## Commonly discussed decision theories

Theories invented by researchers associated with MIRI and LW:

Other decision theories are listed in A comprehensive list of decision theories.

# Can you con­trol the past?

27 Aug 2021 19:39 UTC
170 points

# UDT shows that de­ci­sion the­ory is more puz­zling than ever

13 Sep 2023 12:26 UTC
197 points

# De­ci­sion Theory

31 Oct 2018 18:41 UTC
117 points

# Func­tional De­ci­sion The­ory: A New The­ory of In­stru­men­tal Rationality

20 Oct 2017 8:09 UTC
16 points
(arxiv.org)

# Chris­ti­ano de­ci­sion the­ory excerpt

29 Sep 2019 2:55 UTC
65 points

# Dutch-Book­ing CDT: Re­vised Argument

27 Oct 2020 4:31 UTC
51 points

# An Ortho­dox Case Against Utility Functions

7 Apr 2020 19:18 UTC
152 points

# De­ci­sion The­ory FAQ

28 Feb 2013 14:15 UTC
115 points

# De­ci­sion The­o­ries: A Less Wrong Primer

13 Mar 2012 23:31 UTC
109 points

# Co­her­ence ar­gu­ments do not en­tail goal-di­rected behavior

3 Dec 2018 3:26 UTC
123 points

# Co­her­ent de­ci­sions im­ply con­sis­tent utilities

12 May 2019 21:33 UTC
147 points

# Towards a New De­ci­sion Theory

13 Aug 2009 5:31 UTC
83 points

# Embed­ded Agency (full-text ver­sion)

15 Nov 2018 19:49 UTC
180 points

# LCDT, A My­opic De­ci­sion Theory

3 Aug 2021 22:41 UTC
57 points

# “Do X be­cause de­ci­sion the­ory” ~= “Do X be­cause bayes the­o­rem”

14 Apr 2023 20:57 UTC
39 points

# A Cri­tique of Func­tional De­ci­sion Theory

13 Sep 2019 19:23 UTC
86 points

# Ro­bust Co­op­er­a­tion in the Pri­soner’s Dilemma

7 Jun 2013 8:30 UTC
120 points

# MIRI/​OP ex­change about de­ci­sion theory

25 Aug 2021 22:44 UTC
54 points

# De­ci­sion the­ory and zero-sum game the­ory, NP and PSPACE

24 May 2018 8:03 UTC
56 points

# Three rea­sons to cooperate

24 Dec 2022 17:40 UTC
82 points
(sideways-view.com)

# What is causal­ity to an ev­i­den­tial de­ci­sion the­o­rist?

17 Apr 2022 16:00 UTC
45 points
(sideways-view.com)

# Coun­ter­fac­tual Mug­ging Poker Game

13 Jun 2018 23:34 UTC
111 points

# What does it mean to ap­ply de­ci­sion the­ory?

8 Jul 2020 20:31 UTC
53 points

# [Question] Are ya win­ning, son?

9 Aug 2022 0:06 UTC
14 points

# Re­sponses to ap­par­ent ra­tio­nal­ist con­fu­sions about game /​ de­ci­sion theory

30 Aug 2023 22:02 UTC
138 points

# Troll Bridge

23 Aug 2019 18:36 UTC
79 points

# New­comb’s Prob­lem and Re­gret of Rationality

31 Jan 2008 19:36 UTC
144 points

# Com­ment on Co­her­ence ar­gu­ments do not im­ply goal di­rected behavior

6 Dec 2019 9:30 UTC
30 points

# 5 Ax­ioms of De­ci­sion Making

1 Dec 2011 22:22 UTC
50 points

# Or­di­nary and un­or­di­nary de­ci­sion theory

2 Mar 2022 11:39 UTC
3 points

# De­ci­sion The­ory but also Ghosts

20 Nov 2022 13:24 UTC
17 points

# Model­ing nat­u­ral­ized de­ci­sion prob­lems in lin­ear logic

6 May 2020 0:15 UTC
14 points
(unstableontology.com)

# What I’d change about differ­ent philos­o­phy fields

8 Mar 2021 18:25 UTC
57 points

25 Aug 2009 20:01 UTC
54 points

# Mo­dal Bar­gain­ing Agents

16 Apr 2015 22:19 UTC
14 points

# Why 1-box­ing doesn’t im­ply back­wards causation

25 Mar 2021 2:32 UTC
7 points

# De­ci­sion the­ory: Why we need to re­duce “could”, “would”, “should”

2 Sep 2009 9:23 UTC
36 points

# De­ci­sion the­ory: Why Pearl helps re­duce “could” and “would”, but still leaves us with at least three alternatives

6 Sep 2009 6:10 UTC
43 points

# De­ci­sion The­o­ries: A Semi-For­mal Anal­y­sis, Part I

24 Mar 2012 16:01 UTC
36 points

# The Many Faces of In­fra-Beliefs

6 Apr 2021 10:43 UTC
30 points

# My Cur­rent Take on Counterfactuals

9 Apr 2021 17:51 UTC
53 points

# De­ci­sion The­o­ries: A Semi-For­mal Anal­y­sis, Part II

6 Apr 2012 18:59 UTC
26 points

# De­ci­sion The­o­ries: A Semi-For­mal Anal­y­sis, Part III

14 Apr 2012 19:34 UTC
36 points

# Sav­ing Time

18 May 2021 20:11 UTC
156 points

# The Na­ture of Counterfactuals

5 Jun 2021 9:18 UTC
15 points

# The dumb­est kid in the world (joke)

6 Jun 2021 2:57 UTC
23 points

# I’m no longer sure that I buy dutch book ar­gu­ments and this makes me skep­ti­cal of the “util­ity func­tion” abstraction

22 Jun 2021 3:53 UTC
46 points

# The Joys of Con­ju­gate Priors

21 May 2011 2:41 UTC
63 points

# A Qual­i­ta­tive and In­tu­itive Ex­pla­na­tion of Ex­pected Value

10 Aug 2021 3:31 UTC
11 points

# Meta De­ci­sion The­ory and New­comb’s Problem

5 Mar 2013 1:29 UTC
10 points

# Or­a­cle pre­dic­tions don’t ap­ply to non-ex­is­tent worlds

15 Sep 2021 9:44 UTC
10 points

# Coun­ter­fac­tual Contracts

16 Sep 2021 15:20 UTC
10 points
(harsimony.wordpress.com)

# EDT with up­dat­ing dou­ble counts

12 Oct 2021 4:40 UTC
56 points
(sideways-view.com)

# Prob­a­bil­ity, knowl­edge, and meta-probability

17 Sep 2013 0:02 UTC
58 points

# Ex­ploit­ing New­comb’s Game Show

25 May 2023 4:01 UTC
8 points

# De­ci­sion The­o­ries, Part 3.5: Halt, Melt and Catch Fire

26 Aug 2012 22:40 UTC
49 points

# De­ci­sion The­o­ries, Part 3.75: Hang On, I Think This Works After All

6 Sep 2012 16:23 UTC
39 points

# Nate Soares on the Ul­ti­mate New­comb’s Problem

31 Oct 2021 19:42 UTC
57 points

# [Question] Is Func­tional De­ci­sion The­ory still an ac­tive area of re­search?

13 Nov 2021 0:30 UTC
6 points

# Slightly ad­vanced de­ci­sion the­ory 102: Four rea­sons not to be a (naive) util­ity maximizer

23 Nov 2021 11:02 UTC
10 points
(universalprior.substack.com)

# We need a the­ory of an­thropic mea­sure binding

30 Dec 2021 7:22 UTC
27 points

# De­ci­sion The­ory Break­down—Per­sonal At­tempt at a Review

14 Dec 2021 0:40 UTC
4 points

# Game The­ory with­out Argmax [Part 1]

11 Nov 2023 15:59 UTC
53 points

# \$1000 USD prize—Cir­cu­lar Depen­dency of Counterfactuals

1 Jan 2022 9:43 UTC
37 points

# Game The­ory with­out Argmax [Part 2]

11 Nov 2023 16:02 UTC
31 points

# [Question] What is the sub­jec­tive ex­pe­rience of free will for agents?

2 Apr 2020 15:53 UTC
10 points

# De­ci­sion-the­o­retic prob­lems and The­o­ries; An (In­com­plete) com­par­a­tive list

11 Jul 2018 2:59 UTC
36 points

# Zero-Knowl­edge Cooperation

25 Oct 2017 5:35 UTC
16 points

# Prob­a­bil­ities Small Enough To Ig­nore: An at­tack on Pas­cal’s Mugging

16 Sep 2015 10:45 UTC
27 points

# An ex­ten­sion of Au­mann’s ap­proach for re­duc­ing game the­ory to bayesian de­ci­sion the­ory to in­clude EDT and UDT like agents

9 Feb 2022 4:17 UTC
1 point

# An In­tu­itive In­tro­duc­tion to Ev­i­den­tial De­ci­sion Theory

7 Mar 2022 16:06 UTC
5 points

# UDT can learn an­thropic probabilities

24 Jun 2018 18:04 UTC
54 points

# Time­less Modesty?

24 Nov 2017 11:12 UTC
17 points

# Solve Psy-Kosh’s non-an­thropic problem

20 Dec 2010 21:24 UTC
66 points

# On ex­pected util­ity, part 4: Dutch books, Cox, and Com­plete Class

24 Mar 2022 7:51 UTC
10 points

# Real­ism and Rationality

16 Sep 2019 3:09 UTC
45 points

# CDT=EDT=UDT

13 Jan 2019 23:46 UTC
39 points

# In­fra-Bayesi­anism Distil­la­tion: Real­iz­abil­ity and De­ci­sion Theory

26 May 2022 21:57 UTC
40 points

# [Question] Do FDT (or similar) recom­mend repa­ra­tions?

29 Apr 2022 17:34 UTC
13 points

# [Question] Al­gorith­mic for­mal­iza­tion of FDT?

8 May 2022 1:36 UTC
12 points

# Con­cep­tual Prob­lems with UDT and Policy Selection

28 Jun 2019 23:50 UTC
61 points

# De­ci­sion the­ory and dy­namic inconsistency

3 Jul 2022 22:20 UTC
79 points
(sideways-view.com)

# Im­manuel Kant and the De­ci­sion The­ory App Store

10 Jul 2022 16:04 UTC
88 points

# Mak­ing de­ci­sions us­ing mul­ti­ple worldviews

13 Jul 2022 19:15 UTC
50 points

# FixDT

30 Nov 2023 21:57 UTC
55 points

# De­ci­sion The­ory Para­dox: PD with Three Im­plies Chaos?

27 Aug 2011 19:22 UTC
42 points

# Con­se­quen­tial­ism Need Not Be Nearsighted

2 Sep 2011 7:37 UTC
83 points

# De­ci­sion The­ory Para­dox: An­swer Key

5 Sep 2011 23:13 UTC
10 points

# The Per­spec­tive-based Ex­pla­na­tion to the Reflec­tive In­con­sis­tency Paradox

26 Jan 2024 19:00 UTC
10 points

# Sec­tion 7: Foun­da­tions of Ra­tional Agency

22 Dec 2019 2:05 UTC
14 points

# De­ci­sions are not about chang­ing the world, they are about learn­ing what world you live in

28 Jul 2018 8:41 UTC
39 points

# Less Threat-Depen­dent Bar­gain­ing Solu­tions?? (3/​2)

20 Aug 2022 2:19 UTC
88 points

# Dutch-Book­ing CDT

13 Jan 2019 0:10 UTC
26 points

# [Question] When is CDT Dutch-Book­able?

13 Jan 2019 18:54 UTC
23 points

# On the pur­poses of de­ci­sion the­ory research

25 Jul 2019 7:18 UTC
64 points

# An is­sue with MacAskill’s Ev­i­den­tial­ist’s Wager

21 Sep 2022 22:02 UTC
1 point

# How to Mea­sure Anything

7 Aug 2013 4:05 UTC
118 points

# Refer­ences & Re­sources for LessWrong

10 Oct 2010 14:54 UTC
162 points

# Threat-Re­sis­tant Bar­gain­ing Me­ga­post: In­tro­duc­ing the ROSE Value

28 Sep 2022 1:20 UTC
141 points

# How I Lost 100 Pounds Us­ing TDT

14 Mar 2011 15:50 UTC
127 points

# Mul­ti­verse-wide Co­op­er­a­tion via Cor­re­lated De­ci­sion Making

20 Aug 2017 12:01 UTC
5 points
(foundational-research.org)

# Bounded ver­sions of Gödel’s and Löb’s theorems

27 Jun 2012 18:28 UTC
52 points

# Max­i­mal lot­ter­ies for value learning

16 Oct 2022 23:44 UTC
17 points

# De­ci­sion the­ory does not im­ply that we get to have nice things

18 Oct 2022 3:04 UTC
168 points

# The cor­rect re­sponse to un­cer­tainty is *not* half-speed

15 Jan 2016 22:55 UTC
259 points

# Two More De­ci­sion The­ory Prob­lems for Humans

4 Jan 2019 9:00 UTC
56 points

# Com­mon mis­takes peo­ple make when think­ing about de­ci­sion theory

27 Mar 2012 20:03 UTC
67 points

# In­gre­di­ents of Time­less De­ci­sion Theory

19 Aug 2009 1:10 UTC
52 points

# What is Wei Dai’s Up­date­less De­ci­sion The­ory?

19 May 2010 10:16 UTC
52 points

# Time­less De­ci­sion The­ory: Prob­lems I Can’t Solve

20 Jul 2009 0:02 UTC
56 points

# Com­ment on de­ci­sion theory

9 Sep 2018 20:13 UTC
69 points

# Ba­sic In­framea­sure Theory

27 Aug 2020 8:02 UTC
36 points

# An in­tro­duc­tion to de­ci­sion theory

13 Aug 2010 9:09 UTC
25 points

# New­comb’s Prob­lem: A prob­lem for Causal De­ci­sion Theories

16 Aug 2010 11:25 UTC
11 points

# For­mal­iz­ing New­comb’s

5 Apr 2009 15:39 UTC
22 points

# All About Con­cave and Con­vex Agents

24 Mar 2024 21:37 UTC
59 points

# New­comb II: Newer and Comb-ier

13 Jul 2023 18:49 UTC
0 points

# A short calcu­la­tion about a Twit­ter poll

14 Aug 2023 19:48 UTC
62 points

# De­ci­sions: On­tolog­i­cally Shift­ing to Determinism

21 Dec 2022 12:41 UTC
8 points

# [Question] Coun­ter­fac­tual Mug­ging: Why should you pay?

17 Dec 2019 22:16 UTC
6 points

# Ap­ply­ing the Coun­ter­fac­tual Pri­soner’s Dilemma to Log­i­cal Uncertainty

16 Sep 2020 10:34 UTC
9 points

# Refer­ence Post: For­mal vs. Effec­tive Pre-Commitment

27 Aug 2018 12:04 UTC
16 points

# UDT1.01: The Story So Far (1/​10)

27 Mar 2024 23:22 UTC
31 points

# [Question] What De­ci­sion The­ory is Im­plied By Pre­dic­tive Pro­cess­ing?

28 Sep 2020 17:20 UTC
56 points

# Align­ment work in anoma­lous worlds

16 Dec 2023 19:34 UTC
24 points

# UDT1.01: Lo­cal Affine­ness and In­fluence Mea­sures (2/​10)

31 Mar 2024 7:35 UTC
24 points

# As­sign­ing Praise and Blame: De­cou­pling Episte­mol­ogy and De­ci­sion Theory

27 Jan 2023 18:16 UTC
59 points

# Policy Selec­tion Solves Most Problems

1 Dec 2017 0:35 UTC
21 points

# Mo­dal Fix­point Co­op­er­a­tion with­out Löb’s Theorem

5 Feb 2023 0:58 UTC
133 points

# Log­i­cal Foun­da­tions of Govern­ment Policy

10 Oct 2020 17:05 UTC
2 points

# UDT1.01: Plannable and Un­planned Ob­ser­va­tions (3/​10)

12 Apr 2024 5:24 UTC
31 points

# Pre­dic­tors ex­ist: CDT go­ing bonkers… forever

14 Jan 2020 16:19 UTC
43 points

# ACDT: a hack-y acausal de­ci­sion theory

15 Jan 2020 17:22 UTC
48 points

# Payor’s Lemma in Nat­u­ral Language

2 Mar 2023 12:22 UTC
60 points

# Notes on Prudence

19 Nov 2020 16:14 UTC
14 points

# Treat­ing an­thropic self­ish prefer­ences as an ex­ten­sion of TDT

1 Jan 2015 0:43 UTC
13 points

# Selfish prefer­ences and self-modification

14 Jan 2015 8:42 UTC
12 points

# Kid­nap­ping and the game of Chicken

3 Nov 2013 6:29 UTC
23 points

# [Question] Do agents with (mu­tu­ally known) iden­ti­cal util­ity func­tions but ir­rec­on­cilable knowl­edge some­times fight?

23 Aug 2023 8:13 UTC
14 points

# New­comb Variant

29 Aug 2023 7:02 UTC
25 points

# Any­one want to de­bate pub­li­cly about FDT?

29 Aug 2023 3:45 UTC
13 points

# A mechanis­tic model of meditation

6 Nov 2019 21:37 UTC
130 points

# Strong Cheap Signals

29 Mar 2023 14:18 UTC
29 points
(betonit.substack.com)

# GPT-4 is eas­ily con­trol­led/​ex­ploited with tricky de­ci­sion the­o­retic dilem­mas.

14 Apr 2023 19:39 UTC
6 points

# How LDT helps re­duce the AI arms race

10 Dec 2023 16:21 UTC
70 points

# Alien Axiology

20 Apr 2023 0:27 UTC
3 points

# Refer­ence Post: Triv­ial De­ci­sion The­ory Problem

15 Feb 2020 17:13 UTC
16 points

# The Un­ex­pected Clanging

18 May 2023 14:47 UTC
14 points

# Time in Carte­sian Frames

11 Nov 2020 20:25 UTC
48 points

# What makes coun­ter­fac­tu­als com­pa­rable?

24 Apr 2020 22:47 UTC
11 points

# Con­tra Heighn Con­tra Me Con­tra Func­tional De­ci­sion The­ory

11 Sep 2023 19:49 UTC
−10 points

# Where do self­ish val­ues come from?

18 Nov 2011 23:52 UTC
67 points

# Reflex­ive de­ci­sion the­ory is an un­solved problem

17 Sep 2023 14:15 UTC
39 points

# The om­ni­zoid—Heighn FDT De­bate #5

18 Sep 2023 11:54 UTC
4 points

# For­mal Open Prob­lem in De­ci­sion Theory

29 Nov 2018 3:25 UTC
36 points

# [Question] What causes a de­ci­sion the­ory to be used?

25 Sep 2023 16:33 UTC
8 points

# The Ubiquitous Con­verse Law­vere Problem

29 Nov 2018 3:16 UTC
21 points

# (A → B) → A

11 Sep 2018 22:38 UTC
70 points

# De­ci­sion The­ory with the Magic Parts Highlighted

16 May 2023 17:39 UTC
174 points

# Pri­son­ers’ Dilemma with Costs to Modeling

5 Jun 2018 4:51 UTC
123 points

# De­ci­sion the­ory: An out­line of some up­com­ing posts

25 Aug 2009 7:34 UTC
31 points

# New­comblike prob­lems are the norm

24 Sep 2014 18:41 UTC
83 points

# In­fra-Bayesi­anism Unwrapped

20 Jan 2021 13:35 UTC
54 points

# Com­press­ing Real­ity to Math

15 Dec 2011 0:07 UTC
34 points

# Mea­sures, Risk, Death, and War

20 Dec 2011 23:37 UTC
17 points

# Learn­ing Rus­sian Roulette

2 Apr 2021 18:56 UTC
24 points

# Phy­lac­tery De­ci­sion Theory

2 Apr 2021 20:55 UTC
14 points

# Risk Bud­gets vs. Ba­sic De­ci­sion Theory

5 Apr 2021 21:55 UTC
11 points

# Iden­ti­fi­a­bil­ity Prob­lem for Su­per­ra­tional De­ci­sion Theories

9 Apr 2021 20:33 UTC
17 points

# Defin­ing Myopia

19 Oct 2019 21:32 UTC
32 points

# Naive TDT, Bayes nets, and coun­ter­fac­tual mugging

23 Oct 2012 15:58 UTC
26 points

# Smok­ing le­sion as a coun­terex­am­ple to CDT

26 Oct 2012 12:08 UTC
21 points

# Real-world New­comb-like Prob­lems

25 Mar 2011 20:44 UTC
25 points

# A Ra­tion­al­ity Con­di­tion for CDT Is That It Equal EDT (Part 1)

4 Oct 2018 4:32 UTC
21 points

# A Defense of Func­tional De­ci­sion Theory

12 Nov 2021 20:59 UTC
21 points

# Ques­tion/​Is­sue with the 5/​10 Problem

29 Nov 2021 10:45 UTC
6 points

# Ex­plor­ing De­ci­sion The­o­ries With Coun­ter­fac­tu­als and Dy­namic Agent Self-Pointers

18 Dec 2021 21:50 UTC
2 points

# Wor­ld­build­ing ex­er­cise: The High­way­verse.

22 Dec 2021 6:47 UTC
13 points

# A Re­ac­tion to Wolf­gang Sch­warz’s “On Func­tional De­ci­sion The­ory”

5 Jan 2022 9:00 UTC
7 points

# Cri­tiquing Scasper’s Defi­ni­tion of Sub­junc­tive Dependence

10 Jan 2022 16:22 UTC
6 points

# New­comb’s Lot­tery Problem

27 Jan 2022 16:28 UTC
1 point

# A Pos­si­ble Re­s­olu­tion To Spu­ri­ous Counterfactuals

6 Dec 2021 18:26 UTC
15 points

# Im­pos­si­bil­ity re­sults for un­bounded utilities

2 Feb 2022 3:52 UTC
166 points

# Ba­sic Con­cepts in De­ci­sion Theory

7 Mar 2022 16:05 UTC
3 points

# Defend­ing Func­tional De­ci­sion Theory

8 Feb 2022 14:58 UTC
4 points

# An In­tu­itive In­tro­duc­tion to Causal De­ci­sion Theory

7 Mar 2022 16:05 UTC
3 points

# An In­tu­itive In­tro­duc­tion to Func­tional De­ci­sion Theory

7 Mar 2022 16:07 UTC
19 points

# A Rephras­ing Of and Foot­note To An Embed­ded Agency Proposal

9 Mar 2022 18:13 UTC
5 points

# No, EDT Did Not Get It Right All Along: Why the Coin Flip Creation Prob­lem Is Irrelevant

30 Mar 2022 18:41 UTC
6 points

# Dath Ilani Rule of Law

10 May 2022 6:17 UTC
18 points

# Open Prob­lems with Myopia

10 Mar 2021 18:38 UTC
65 points

# Unify­ing Bar­gain­ing No­tions (1/​2)

25 Jul 2022 0:28 UTC
204 points

# Wanted: No­ta­tion for credal resilience

31 Jul 2022 7:35 UTC
21 points

# [Question] How would Log­i­cal De­ci­sion The­o­ries ad­dress the Psy­chopath But­ton?

7 Aug 2022 15:19 UTC
5 points

# [Question] How would two su­per­in­tel­li­gent AIs in­ter­act, if they are un­al­igned with each other?

9 Aug 2022 18:58 UTC
4 points

# [Question] Do ad­vance­ments in De­ci­sion The­ory point to­wards moral ab­solutism?

11 Aug 2022 0:59 UTC
0 points

# Bridg­ing Ex­pected Utility Max­i­miza­tion and Optimization

5 Aug 2022 8:18 UTC
25 points

# [Question] Perfect Predictors

12 Aug 2022 11:51 UTC
2 points

# Dis­cov­er­ing Agents

18 Aug 2022 17:33 UTC
73 points

# Ini­tial Thoughts on Dis­solv­ing “Could­ness”

22 Sep 2022 21:23 UTC
6 points

# Break­ing New­comb’s Prob­lem with Non-Halt­ing states

4 Sep 2022 4:01 UTC
18 points

# Un­bounded util­ity func­tions and precommitment

10 Sep 2022 16:16 UTC
4 points

# FDT defects in a re­al­is­tic Twin Pri­son­ers’ Dilemma

15 Sep 2022 8:55 UTC
37 points

# I’m tak­ing a course on game the­ory and am faced with this ques­tion. What’s the ra­tio­nal de­ci­sion?

14 Sep 2022 0:27 UTC
0 points

# Train­ing goals for large lan­guage models

18 Jul 2022 7:09 UTC
28 points

# An Un­ex­pected GPT-3 De­ci­sion in a Sim­ple Gam­ble

25 Sep 2022 16:46 UTC
8 points

# FDT is not di­rectly com­pa­rable to CDT and EDT

29 Sep 2022 14:42 UTC
36 points

# [Sketch] Val­idity Cri­te­rion for Log­i­cal Counterfactuals

11 Oct 2022 13:31 UTC
6 points

# Notes on “Can you con­trol the past”

20 Oct 2022 3:41 UTC
60 points

# Log­i­cal De­ci­sion The­o­ries: Our fi­nal failsafe?

25 Oct 2022 12:51 UTC
−7 points
(www.lesswrong.com)

# Hu­mans do acausal co­or­di­na­tion all the time

2 Nov 2022 14:40 UTC
57 points

# Fur­ther con­sid­er­a­tions on the Ev­i­den­tial­ist’s Wager

3 Nov 2022 20:06 UTC
3 points

# Ad­ver­sar­ial Pri­ors: Not Pay­ing Peo­ple to Lie to You

10 Nov 2022 2:29 UTC
22 points

# De­ci­sion mak­ing un­der model am­bi­guity, moral un­cer­tainty, and other agents with free will?

13 Nov 2022 12:50 UTC
3 points
(forum.effectivealtruism.org)

# Two New New­comb Variants

14 Nov 2022 14:01 UTC
26 points

# SBF x LoL

15 Nov 2022 20:24 UTC
17 points

# SBF, Pas­cal’s Mug­ging, and a Pro­posed Solution

18 Nov 2022 18:39 UTC
−1 points
(colekillian.com)

# Fair Col­lec­tive Effi­cient Altruism

25 Nov 2022 9:38 UTC
2 points

# Why Bet Kelly?

29 Nov 2022 18:47 UTC
16 points

# Con­di­tions for Su­per­ra­tional­ity-mo­ti­vated Co­op­er­a­tion in a one-shot Pri­soner’s Dilemma

19 Dec 2022 15:00 UTC
24 points

# A prob­lem with “play­ing chicken with the uni­verse” as an ap­proach to UDT

8 Mar 2013 2:34 UTC
35 points

# Mo­ral strate­gies at differ­ent ca­pa­bil­ity levels

27 Jul 2022 18:50 UTC
112 points
(thinkingcomplete.blogspot.com)

# You’re Not One “You”—How De­ci­sion The­o­ries Are Talk­ing Past Each Other

9 Jan 2023 1:21 UTC
27 points

# Proper scor­ing rules don’t guaran­tee pre­dict­ing fixed points

16 Dec 2022 18:22 UTC
68 points

# What can thought-ex­per­i­ments do?

17 Jan 2023 0:35 UTC
16 points

# Threat­en­ing to do the im­pos­si­ble: A solu­tion to spu­ri­ous coun­ter­fac­tu­als for func­tional de­ci­sion the­ory via proof theory

11 Feb 2023 7:57 UTC
5 points

# Heuris­tics on bias to ac­tion ver­sus sta­tus quo?

28 Feb 2023 12:45 UTC
4 points

# Some Var­i­ants of Sleep­ing Beauty

1 Mar 2023 16:51 UTC
34 points

# Don’t Jump or I’ll...

2 Mar 2023 2:58 UTC
13 points

# Hert­ford, Sour­but (ra­tio­nal­ity les­sons from Univer­sity Challenge)

4 Sep 2023 18:44 UTC
28 points
(www.oliversourbut.net)

# [Question] Why does ex­pected util­ity mat­ter?

25 Dec 2023 14:47 UTC
18 points

# So­cial Choice The­ory and Log­i­cal Handshakes

29 Dec 2023 3:49 UTC
14 points

# Us­ing Threats to Achieve So­cially Op­ti­mal Outcomes

4 Jan 2024 23:30 UTC
8 points

# Best-Re­spond­ing Is Not Always the Best Response

4 Jan 2024 23:30 UTC
10 points

# In defense of an­throp­i­cally up­dat­ing EDT

5 Mar 2024 6:21 UTC
17 points

# A sur­vey of polls on New­comb’s problem

20 Sep 2017 16:50 UTC
3 points
(casparoesterheld.com)

# A Bench­mark for De­ci­sion Theories

11 Jan 2024 18:54 UTC
10 points

# Even if we lose, we win

15 Jan 2024 2:15 UTC
23 points

# In­cor­po­rat­ing Jus­tice The­ory into De­ci­sion Theory

21 Jan 2024 19:17 UTC
13 points

# Refram­ing Acausal Trol­ling as Acausal Patronage

23 Jan 2024 3:04 UTC
14 points

# Dis­tance Func­tions are Hard

13 Aug 2019 17:33 UTC
31 points

# To Boldly Code

26 Jan 2024 18:25 UTC
25 points

# Coun­ter­fac­tual Mechanism Networks

30 Jan 2024 20:30 UTC
4 points

26 Jan 2024 18:25 UTC
17 points

# [Question] How to deal with the sense of de­mo­ti­va­tion that comes from think­ing about de­ter­minism?

7 Feb 2024 10:53 UTC
13 points

# Up­date­less­ness doesn’t solve most problems

8 Feb 2024 17:30 UTC
124 points

# The lat­tice of par­tial updatelessness

10 Feb 2024 17:34 UTC
21 points

# Storable Votes with a Pay as you win mechanism: a con­tri­bu­tion for in­sti­tu­tional design

11 Mar 2024 15:58 UTC
17 points

# Ex­plicit Op­ti­miza­tion of Global Strat­egy (Fix­ing a Bug in UDT1)

19 Feb 2010 1:30 UTC
55 points

# The Bind­ing of Isaac & Trans­par­ent New­comb’s Prob­lem

22 Feb 2024 18:56 UTC
−11 points

# [Question] CDT vs. EDT on Deterrence

24 Feb 2024 15:41 UTC
1 point

# Co­op­er­at­ing with aliens and AGIs: An ECL explainer

24 Feb 2024 22:58 UTC
53 points

# Everett branches, in­ter-light cone trade and other alien mat­ters: Ap­pendix to “An ECL ex­plainer”

24 Feb 2024 23:09 UTC
17 points

# Delta’s of Change

19 Mar 2024 21:03 UTC
1 point

# GPT-4 al­ign­ing with aca­sual de­ci­sion the­ory when in­structed to play games, but in­cludes a CDT ex­pla­na­tion that’s in­cor­rect if they differ

23 Mar 2023 16:16 UTC
7 points

# “Don’t even think about hell”

2 May 2020 8:06 UTC
6 points

# Mo­ral­ity vs re­lated concepts

7 Jan 2020 10:47 UTC
26 points

# Mo­ral un­cer­tainty vs re­lated concepts

11 Jan 2020 10:03 UTC
26 points

# Mak­ing de­ci­sions when both morally and em­piri­cally uncertain

2 Jan 2020 7:20 UTC
13 points

# Mak­ing de­ci­sions un­der moral uncertainty

30 Dec 2019 1:49 UTC
20 points

# Mo­ral un­cer­tainty: What kind of ‘should’ is in­volved?

13 Jan 2020 12:13 UTC
14 points

# Dis­solv­ing Con­fu­sion around Func­tional De­ci­sion Theory

5 Jan 2020 6:38 UTC
32 points

# Disen­tan­gling four mo­ti­va­tions for act­ing in ac­cor­dance with UDT

5 Nov 2023 21:26 UTC
33 points

# Im­ple­ment­ing De­ci­sion Theory

7 Nov 2023 17:55 UTC
21 points

# An­throp­i­cal Para­doxes are Para­doxes of Prob­a­bil­ity Theory

6 Dec 2023 8:16 UTC
49 points

# Pre­dictable Defect-Co­op­er­ate?

18 Nov 2023 15:38 UTC
7 points

# Self-Refer­en­tial Prob­a­bil­is­tic Logic Ad­mits the Payor’s Lemma

28 Nov 2023 10:27 UTC
80 points

# [Question] 3-P Group op­ti­mal for dis­cus­sion?

13 Jul 2020 22:23 UTC
3 points

# Reflec­tive con­sis­tency, ran­dom­ized de­ci­sions, and the dan­gers of un­re­al­is­tic thought experiments

7 Dec 2023 3:33 UTC
34 points

# Coun­ter­fac­tual Re­pro­gram­ming De­ci­sion Theory

10 Sep 2012 1:35 UTC
18 points

# Beyond Astro­nom­i­cal Waste

7 Jun 2018 21:04 UTC
126 points

# Prob­lems in AI Align­ment that philoso­phers could po­ten­tially con­tribute to

17 Aug 2019 17:38 UTC
78 points

# The Dar­win Results

25 Nov 2017 13:30 UTC
52 points
(thezvi.wordpress.com)

# Im­plicit extortion

13 Apr 2018 16:33 UTC
29 points
(ai-alignment.com)

# Sim­plified Poker

4 Jun 2018 15:50 UTC
34 points
(thezvi.wordpress.com)

# Against the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Penalty

14 Dec 2017 18:38 UTC
39 points

# Pavlov Generalizes

20 Feb 2019 9:03 UTC
65 points

# TDT for Humans

28 Feb 2018 5:40 UTC
26 points

# EDT solves 5 and 10 with con­di­tional oracles

30 Sep 2018 7:57 UTC
59 points

# Bayesi­ans vs. Barbarians

14 Apr 2009 23:45 UTC
100 points

# Com­par­i­son of de­ci­sion the­o­ries (with a fo­cus on log­i­cal-coun­ter­fac­tual de­ci­sion the­o­ries)

16 Mar 2019 21:15 UTC
77 points

# Pas­cal’s Mug­ging: Tiny Prob­a­bil­ities of Vast Utilities

19 Oct 2007 23:37 UTC
105 points

# “UDT2” and “against UD+ASSA”

12 May 2019 4:18 UTC
50 points

# You May Already Be A Sinner

9 Mar 2009 23:18 UTC
50 points

# Pas­cal’s Mug­gle Pays

16 Dec 2017 20:40 UTC
25 points
(thezvi.wordpress.com)

# Coun­ter­fac­tual Mugging

19 Mar 2009 6:08 UTC
80 points

# Pas­cal’s Mug­gle: In­finites­i­mal Pri­ors and Strong Evidence

8 May 2013 0:43 UTC
72 points

# Avert­ing Catas­tro­phe: De­ci­sion The­ory for COVID-19, Cli­mate Change, and Po­ten­tial Disasters of All Kinds

2 May 2023 22:50 UTC
10 points

21 Apr 2012 13:33 UTC
70 points

# A model of UDT with a con­crete prior over log­i­cal statements

28 Aug 2012 21:45 UTC
62 points

# New­comb’s prob­lem hap­pened to me

26 Mar 2010 18:31 UTC
56 points

# (Ir)ra­tio­nal­ity of Pas­cal’s wager

3 Aug 2020 20:57 UTC
3 points

# A model of UDT with a halt­ing oracle

18 Dec 2011 14:18 UTC
68 points

# [Question] Is EDT cor­rect? Does “EDT” == “log­i­cal EDT” == “log­i­cal CDT”?

8 May 2023 2:07 UTC
13 points

# More on the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Prior

26 Feb 2018 23:53 UTC
16 points

# Acausal trade nat­u­rally re­sults in the Nash bar­gain­ing solution

8 May 2023 18:13 UTC
3 points

# What a re­duc­tion of “could” could look like

12 Aug 2010 17:41 UTC
83 points

# Knowl­edge is Freedom

9 Feb 2018 5:24 UTC
32 points

# Log­i­cal Up­date­less­ness as a Ro­bust Del­e­ga­tion Problem

27 Oct 2017 21:16 UTC
38 points

# Parfit’s Es­cape (Filk)

29 Mar 2019 2:31 UTC
39 points

# The Black­mail Equation

10 Mar 2010 14:46 UTC
27 points

# [Question] Can we learn much by study­ing the be­havi­our of RL poli­cies?

15 May 2023 12:56 UTC
1 point

# Two Types of Updatelessness

15 Feb 2018 20:19 UTC
23 points

# Two Alter­na­tives to Log­i­cal Counterfactuals

1 Apr 2020 9:48 UTC
38 points
(unstableontology.com)

# Policy Alignment

30 Jun 2018 0:24 UTC
50 points

# Is risk aver­sion re­ally ir­ra­tional ?

31 Jan 2012 20:34 UTC
54 points

# Oper­a­tional­iz­ing New­comb’s Problem

11 Nov 2019 22:52 UTC
34 points

# Another at­tempt to ex­plain UDT

14 Nov 2010 16:52 UTC
69 points

# The Happy Dance Problem

17 Nov 2017 0:47 UTC
19 points

# UDT as a Nash Equilibrium

6 Feb 2018 14:08 UTC
18 points

# A prob­lem with Time­less De­ci­sion The­ory (TDT)

4 Feb 2010 18:47 UTC
46 points

# Solv­ing the two en­velopes problem

9 Aug 2012 13:42 UTC
45 points

# Prob­le­matic Prob­lems for TDT

29 May 2012 15:41 UTC
62 points

# The Ab­sent-Minded Driver

16 Sep 2009 0:51 UTC
45 points

# Com­plete Class: Con­se­quen­tial­ist Foundations

11 Jul 2018 1:57 UTC
53 points

# “Cheat­ing Death in Da­m­as­cus” Solu­tion to the Fermi Para­dox

30 Jun 2018 12:00 UTC
14 points

# Self-Similar­ity Experiment

15 Aug 2020 13:19 UTC
12 points

# Let’s Dis­cuss Func­tional De­ci­sion Theory

23 Jul 2018 7:24 UTC
29 points

# List of Prob­lems That Mo­ti­vated UDT

6 Jun 2012 0:26 UTC
42 points

# [Question] A way to beat su­per­ra­tional/​EDT agents?

17 Aug 2020 14:33 UTC
5 points

# The Pre­dic­tion Prob­lem: A Var­i­ant on New­comb’s

4 Jul 2018 7:40 UTC
25 points

# L-zom­bies! (L-zom­bies?)

7 Feb 2014 18:30 UTC
52 points

# Why you must max­i­mize ex­pected utility

13 Dec 2012 1:11 UTC
50 points

# Mixed-Strat­egy Rat­ifi­a­bil­ity Im­plies CDT=EDT

31 Oct 2017 5:56 UTC
12 points

# An ex­pla­na­tion of de­ci­sion theories

1 Jun 2023 3:42 UTC
20 points

# Four lev­els of un­der­stand­ing de­ci­sion theory

1 Jun 2023 20:55 UTC
12 points

# [Question] Which text­book would you recom­mend to learn de­ci­sion the­ory?

29 Jan 2019 20:48 UTC
27 points

# A Prob­lem About Bar­gain­ing and Log­i­cal Uncertainty

21 Mar 2012 21:03 UTC
47 points

# What Pro­gram Are You?

12 Oct 2009 0:29 UTC
36 points

# Time­less De­ci­sion The­ory and Meta-Cir­cu­lar De­ci­sion Theory

20 Aug 2009 22:07 UTC
41 points

# Asymp­totic De­ci­sion The­ory (Im­proved Wri­teup)

27 Sep 2018 5:17 UTC
39 points

# AI co­op­er­a­tion in practice

30 Jul 2010 16:21 UTC
45 points

# For­mu­las of ar­ith­metic that be­have like de­ci­sion agents

3 Feb 2012 2:58 UTC
35 points

# Con­trol­ling Con­stant Programs

5 Sep 2010 13:45 UTC
34 points

# In­tro­duc­ing The Long Game Pro­ject: Im­prov­ing De­ci­sion-Mak­ing Through Table­top Ex­er­cises and Si­mu­lated Experience

13 Jun 2023 21:45 UTC
4 points

# Does TDT pay in Coun­ter­fac­tual Mug­ging?

29 Nov 2010 21:31 UTC
4 points

# An­throp­i­cally Blind: the an­thropic shadow is re­flec­tively inconsistent

29 Jun 2023 2:36 UTC
40 points

# Ex­am­ple de­ci­sion the­ory prob­lem: “Agent simu­lates pre­dic­tor”

19 May 2011 15:16 UTC
45 points

# Quan­tum ver­sus log­i­cal bombs

17 Nov 2013 15:14 UTC
27 points

# Quan­tum im­mor­tal­ity: Is de­cline of mea­sure com­pen­sated by merg­ing timelines?

11 Dec 2018 19:39 UTC
9 points

# Quan­tum the­ory can­not con­sis­tently de­scribe the use of itself

20 Sep 2018 22:04 UTC
7 points
(foundations.ethz.ch)

# Quan­tum Rus­sian Roulette

18 Sep 2009 8:49 UTC
8 points

# Knigh­tian un­cer­tainty: a re­jec­tion of the MMEU rule

26 Aug 2014 3:03 UTC
41 points

# Ex­plor­ing Func­tional De­ci­sion The­ory (FDT) and a mod­ified ver­sion (ModFDT)

5 Jul 2023 14:06 UTC
8 points

# Open-minded updatelessness

10 Jul 2023 11:08 UTC
65 points

# Ev­i­den­tial De­ci­sion The­ory, Selec­tion Bias, and Refer­ence Classes

8 Jul 2013 5:16 UTC
32 points

# Philo­soph­i­cal self-ratification

3 Feb 2020 22:48 UTC
23 points
(unstableontology.com)

# For­mal­is­ing de­ci­sion the­ory is hard

23 Aug 2019 3:27 UTC
17 points

# Build a Causal De­ci­sion Theorist

9 Mar 2023 13:31 UTC
−2 points

# Proofs Sec­tion 2.3 (Up­dates, De­ci­sion The­ory)

27 Aug 2020 7:49 UTC
8 points

# Proofs Sec­tion 2.2 (Iso­mor­phism to Ex­pec­ta­tions)

27 Aug 2020 7:52 UTC
8 points

# Proofs Sec­tion 2.1 (The­o­rem 1, Lem­mas)

27 Aug 2020 7:54 UTC
8 points

# In­tro­duc­tion To The In­fra-Bayesi­anism Sequence

26 Aug 2020 20:31 UTC
108 points

# Belief Func­tions And De­ci­sion Theory

27 Aug 2020 8:00 UTC
17 points

# The Ul­ti­mate New­comb’s Problem

10 Sep 2013 2:03 UTC
46 points

# Re­port on mod­el­ing ev­i­den­tial co­op­er­a­tion in large worlds

12 Jul 2023 16:37 UTC
44 points
(arxiv.org)

# Op­ti­mi­sa­tion Mea­sures: Desider­ata, Im­pos­si­bil­ity, Proposals

7 Aug 2023 15:52 UTC
35 points

# Acausal Now: We could to­tally acausally bar­gain with aliens at our cur­rent tech level if desired

9 Aug 2023 0:50 UTC
1 point

# A full ex­pla­na­tion to New­comb’s para­dox.

12 Oct 2020 16:48 UTC
−6 points

# The Achilles Heel Hy­poth­e­sis for AI

13 Oct 2020 14:35 UTC
20 points

# Knigh­tian Uncer­tainty and Am­bi­guity Aver­sion: Motivation

21 Jul 2014 20:32 UTC
44 points

# Thoughts from a Two Boxer

23 Aug 2019 0:24 UTC
18 points

# The Evil Ge­nie Puzzle

25 Jul 2018 6:12 UTC
18 points

# Im­pli­ca­tions of ev­i­den­tial co­op­er­a­tion in large worlds

23 Aug 2023 0:43 UTC
39 points
(lukasfinnveden.substack.com)

# In mem­o­ryless Carte­sian en­vi­ron­ments, ev­ery UDT policy is a CDT+SIA policy

11 Jun 2016 4:05 UTC
25 points

# [Question] What’s been writ­ten about the na­ture of “son-of-CDT”?

30 Nov 2019 21:03 UTC
16 points

# Win­ning is Hard

3 Apr 2009 17:02 UTC
−10 points

# Ra­tional Agents Co­op­er­ate in the Pri­soner’s Dilemma

2 Sep 2023 6:15 UTC
17 points

# De­ci­sion the­ory is not policy the­ory is not agent theory

5 Sep 2023 1:38 UTC
15 points
(colewyeth.com)

# De­ci­sion The­ory: A (Nor­ma­tive) Introduction

6 Sep 2023 8:22 UTC
−1 points
(paretooptimal.substack.com)

# [Question] Is Agent Si­mu­lates Pre­dic­tor a “fair” prob­lem?

24 Jan 2019 13:18 UTC
22 points

# A New Bayesian De­ci­sion Theory

20 Sep 2023 9:36 UTC
−6 points
(paretooptimal.substack.com)

# An ex­am­ple of self-fulfilling spu­ri­ous proofs in UDT

25 Mar 2012 11:47 UTC
33 points

# Thoughts on the 5-10 Problem

18 Jul 2019 18:56 UTC
19 points

# Why We Use Money? - A Walrasian View

3 Oct 2023 12:02 UTC
4 points

# Ar­gu­ments for util­i­tar­i­anism are im­pos­si­bil­ity ar­gu­ments un­der un­bounded prospects

7 Oct 2023 21:08 UTC
7 points

# Per­spec­tive Based Rea­son­ing Could Ab­solve CDT

8 Oct 2023 11:22 UTC
4 points

# UDT might not pay a Coun­ter­fac­tual Mugger

21 Nov 2020 23:27 UTC
5 points

# Shane Legg on prospect the­ory and com­pu­ta­tional finance

21 Jun 2009 17:57 UTC
16 points

# The ap­pli­ca­tion of the sec­re­tary prob­lem to real life dating

29 Sep 2015 22:28 UTC
7 points

# The Fixed Sum Fallacy

3 Jul 2009 13:01 UTC
5 points

# Which Anaes­thetic To Choose?

14 Oct 2023 14:55 UTC
10 points

# How Less­wrong helped me make \$25K: A ra­tio­nal pric­ing strategy

21 Dec 2020 20:20 UTC
50 points

# Coun­ter­fac­tual Plan­ning in AGI Systems

3 Feb 2021 13:54 UTC
10 points

# Graph­i­cal World Models, Coun­ter­fac­tu­als, and Ma­chine Learn­ing Agents

17 Feb 2021 11:07 UTC
6 points

# A non-log­a­r­ith­mic ar­gu­ment for Kelly

4 Mar 2021 16:21 UTC
24 points