Pit­falls of Rationality

TagLast edit: 26 Jul 2020 0:21 UTC by Ruby

Pit­falls of Ra­tion­al­ity are fre­quent er­ror modes, ob­sta­cles or prob­lems that arise when peo­ple try to prac­tice ra­tio­nal­ity, or en­gage with ra­tio­nal­ity-re­lated ma­te­ri­als. Re­lated con­cepts in­clude the “valley of bad ra­tio­nal­ity”.

There are two threads touched in posts un­der this tag:

  1. Things that go wrong when peo­ple try to be more ra­tio­nal and they un­in­ten­tion­ally end up mak­ing things worse.

  2. Ar­guably, why haven’t ra­tio­nal­ists visi­ble suc­ceeded at their bold and am­bi­tious goals yet?

Re­gard­ing the first point, from In­cre­men­tal Progress and the Valley:

Ah. Well, here’s the the thing: An in­cre­men­tal step in the di­rec­tion of ra­tio­nal­ity, if the re­sult is still ir­ra­tional in other ways, does not have to yield in­cre­men­tally more win­ning.

The op­ti­mal­ity the­o­rems that we have for prob­a­bil­ity the­ory and de­ci­sion the­ory, are for perfect prob­a­bil­ity the­ory and de­ci­sion the­ory. There is no com­pan­ion the­o­rem which says that, start­ing from some flawed ini­tial form, ev­ery in­cre­men­tal mod­ifi­ca­tion of the al­gorithm that takes the struc­ture closer to the ideal, must yield an in­cre­men­tal im­prove­ment in perfor­mance. This has not yet been proven, be­cause it is not, in fact, true.

Know­ing About Bi­ases Can Hurt People

Eliezer Yudkowsky4 Apr 2007 18:01 UTC
119 points
80 comments2 min readLW link

Real­ity-Re­veal­ing and Real­ity-Mask­ing Puzzles

AnnaSalamon16 Jan 2020 16:15 UTC
252 points
58 comments13 min readLW link

Mak­ing your ex­plicit rea­son­ing trust­wor­thy

AnnaSalamon29 Oct 2010 0:00 UTC
92 points
95 comments6 min readLW link

Self-Im­prove­ment or Shiny Dis­trac­tion: Why Less Wrong is anti-In­stru­men­tal Rationality

patrissimo14 Sep 2010 16:17 UTC
119 points
261 comments16 min readLW link

Ex­treme Ra­tion­al­ity: It’s Not That Great

Scott Alexander9 Apr 2009 2:44 UTC
175 points
276 comments8 min readLW link

My ex­pe­rience with the “ra­tio­nal­ist un­canny valley”

Thomas Kwa23 Apr 2020 20:27 UTC
62 points
18 comments5 min readLW link

“Epiphany ad­dic­tion”

cousin_it3 Aug 2012 17:52 UTC
52 points
94 comments1 min readLW link

Sum­mary of “The Straw Vul­can”

alexvermeer26 Dec 2011 16:29 UTC
36 points
27 comments6 min readLW link

The Treach­er­ous Path to Rationality

Jacobian9 Oct 2020 15:34 UTC
152 points
110 comments11 min readLW link

No Really, Why Aren’t Ra­tion­al­ists Win­ning?

Sailor Vulcan4 Nov 2018 18:11 UTC
30 points
85 comments5 min readLW link

Why Don’t Ra­tion­al­ists Win?

adamzerner5 Sep 2015 0:57 UTC
1 point
118 comments6 min readLW link


Chris_Leong6 Feb 2018 8:25 UTC
54 points
18 comments2 min readLW link

Trust in Bayes

Eliezer Yudkowsky29 Jan 2008 23:12 UTC
30 points
27 comments8 min readLW link

Ad­ding Up To Normality

orthonormal24 Mar 2020 21:53 UTC
70 points
20 comments3 min readLW link

Map Er­rors: The Good, The Bad, and The Territory

orthonormal27 Jun 2020 5:22 UTC
25 points
4 comments2 min readLW link

Don’t Make Your Prob­lems Hide

orthonormal27 Jun 2020 20:24 UTC
52 points
5 comments2 min readLW link

In­cre­men­tal Progress and the Valley

Eliezer Yudkowsky4 Apr 2009 16:42 UTC
61 points
113 comments5 min readLW link

Bayesi­ans vs. Barbarians

Eliezer Yudkowsky14 Apr 2009 23:45 UTC
64 points
283 comments8 min readLW link

Ra­tion­al­ists should be­ware rationalism

Kaj_Sotala6 Apr 2009 14:16 UTC
27 points
32 comments3 min readLW link

In praise of heuristics

Bucky24 Oct 2018 15:44 UTC
44 points
27 comments7 min readLW link

Ac­cu­racy Ver­sus Winning

John_Maxwell2 Apr 2009 4:47 UTC
12 points
77 comments1 min readLW link

The Costs of Rationality

RobinHanson3 Mar 2009 18:13 UTC
35 points
81 comments1 min readLW link

Mis­takes with Con­ser­va­tion of Ex­pected Evidence

abramdemski8 Jun 2019 23:07 UTC
148 points
15 comments12 min readLW link

Don’t Pull a Bro­ken Chain

johnswentworth28 Aug 2019 1:21 UTC
27 points
6 comments5 min readLW link

18-month fol­low-up on my self-con­cept work

Kaj_Sotala18 Dec 2018 17:40 UTC
57 points
4 comments8 min readLW link

What it means to optimise

Neel Nanda25 Jul 2020 9:40 UTC
7 points
0 comments8 min readLW link

“No ev­i­dence” as a Valley of Bad Rationality

adamzerner28 Mar 2020 23:45 UTC
108 points
14 comments2 min readLW link

The prob­lem with too many ra­tio­nal memes

Swimmer96319 Jan 2012 0:56 UTC
87 points
342 comments3 min readLW link

Dou­ble-Dip­ping in Dun­ning—Kruger

isovector28 Nov 2018 3:40 UTC
76 points
31 comments3 min readLW link

Against Shoot­ing Your­self in the Foot

Eliezer Yudkowsky16 Nov 2017 20:13 UTC
66 points
3 comments3 min readLW link

An Out­side View on Less Wrong’s Advice

Mass_Driver7 Jul 2011 4:46 UTC
64 points
160 comments8 min readLW link

Cul­tish Countercultishness

Eliezer Yudkowsky30 Dec 2007 0:53 UTC
51 points
30 comments9 min readLW link

Poll re­sults: LW prob­a­bly doesn’t cause akrasia

AnnaSalamon16 Nov 2011 18:03 UTC
47 points
109 comments8 min readLW link

Sim­pli­cio and Sophisticus

Zvi22 Jul 2018 13:30 UTC
42 points
1 comment4 min readLW link

I’m scared.

Mass_Driver23 Dec 2010 9:05 UTC
41 points
81 comments1 min readLW link

Be­ing Half-Ra­tional About Pas­cal’s Wager is Even Worse

Eliezer Yudkowsky18 Apr 2013 5:20 UTC
29 points
166 comments9 min readLW link

Goals for which Less Wrong does (and doesn’t) help

AnnaSalamon18 Nov 2010 22:37 UTC
66 points
104 comments4 min readLW link

The Sin of Underconfidence

Eliezer Yudkowsky20 Apr 2009 6:30 UTC
67 points
185 comments6 min readLW link

Defect­ing by Ac­ci­dent—A Flaw Com­mon to An­a­lyt­i­cal People

lionhearted1 Dec 2010 8:25 UTC
106 points
431 comments15 min readLW link

Too Smart for My Own Good

isovector22 Jan 2019 17:51 UTC
40 points
4 comments3 min readLW link

Self-Con­grat­u­la­tory Rationalism

ChrisHallquist1 Mar 2014 8:52 UTC
53 points
395 comments10 min readLW link

Fire­wal­ling the Op­ti­mal from the Rational

Eliezer Yudkowsky8 Oct 2012 8:01 UTC
104 points
337 comments5 min readLW link

Sto­icism: Cau­tion­ary Advice

VivaLaPanda14 Nov 2018 23:18 UTC
37 points
16 comments3 min readLW link

Truth: It’s Not That Great

ChrisHallquist4 May 2014 22:07 UTC
37 points
58 comments4 min readLW link

Take heed, for it is a trap

Zed14 Aug 2011 10:23 UTC
47 points
189 comments6 min readLW link

Se­duced by Imagination

Eliezer Yudkowsky16 Jan 2009 3:10 UTC
25 points
20 comments3 min readLW link

De-Cen­ter­ing Bias

Chris_Leong18 Oct 2017 23:24 UTC
28 points
10 comments2 min readLW link

[Link] Nerds are nuts

[deleted]7 Jun 2012 7:48 UTC
25 points
44 comments8 min readLW link

Rea­son as memetic im­mune disorder

PhilGoetz19 Sep 2009 21:05 UTC
294 points
175 comments5 min readLW link

Memes and Ra­tional Decisions

inferential9 Jan 2015 6:42 UTC
25 points
17 comments10 min readLW link

Do we have it too easy?

Klao2 Nov 2011 1:53 UTC
27 points
44 comments3 min readLW link

I’m be­com­ing in­tol­er­ant. Help.

loup-vaillant30 Jun 2011 15:30 UTC
26 points
87 comments1 min readLW link

Skills and Antiskills

katydee26 Apr 2014 6:54 UTC
31 points
54 comments1 min readLW link

A Re­fu­ta­tion of (Global) “Hap­piness Max­i­miza­tion”

fare25 Aug 2020 20:33 UTC
−3 points
4 comments15 min readLW link
No comments.