Thinking and Deciding: a chapter by chapter review

This is a chap­ter-by-chap­ter re­view of Think­ing and De­cid­ing by Jonathan Baron (UPenn, twit­ter). It won’t be a de­tailed sum­mary like bad­ger’s ex­cel­lent sum­mary of Episte­mol­ogy and the Psy­chol­ogy of Hu­man Judg­ment, in part be­cause this is a 600-page text­book and so a full sum­mary would be far longer that I want to write here. I’ll try to provide enough de­tails that peo­ple can seek out the chap­ters that they find in­ter­est­ing, but this is by no means a re­place­ment for read­ing the chap­ters that you find in­ter­est­ing. Every chap­ter is dis­cussed be­low, with a brief “what should I read?” sec­tion if you know what you’re in­ter­ested in.

We already have a thread for text­book recom­men­da­tions, but this book is cen­tral enough to Less Wrong’s mis­sion that it seems like it’s worth an in-depth re­view. I’ll state my ba­sic im­pres­sion of the whole book up front: I ex­pect most read­ers of LW would gain quite a bit from read­ing the book, es­pe­cially newer mem­bers, as it seems like a more fo­cused and bal­anced in­tro­duc­tion to the sub­ject of ra­tio­nal­ity than the Se­quences.

Baron splits the book into three sec­tions: Think­ing in Gen­eral, Prob­a­bil­ity and Belief, and De­ci­sions and Plans.

I may as well quote the first page in its en­tirety, as I feel it gives a good de­scrip­tion of the book:

Begin­ning with its first edi­tion and through three sub­se­quent edi­tions, Think­ing and De­cid­ing has es­tab­lished it­self as the re­quired text and im­por­tant refer­ence work for stu­dents and schol­ars of hu­man cog­ni­tion and ra­tio­nal­ity. In this, the fourth edi­tion, Jonathan Baron re­tains the com­pre­hen­sive at­ten­tion to the key ques­tions ad­dressed in pre­vi­ous edi­tions- How should we think? What, if any­thing, keeps us from think­ing that way? How can we im­prove our think­ing and de­ci­sion mak­ing? - and his ex­panded treat­ment of top­ics such as risk, util­i­tar­i­anism, Bayes’s the­o­rem, and moral think­ing. With the stu­dent in mind, the fourth edi­tion em­pha­sizes the de­vel­op­ment of an un­der­stand­ing of the fun­da­men­tal con­cepts in judg­ment and de­ci­sion mak­ing. This book is es­sen­tial read­ing for stu­dents and schol­ars in judg­ment and de­ci­sion mak­ing and re­lated fields, in­clud­ing psy­chol­ogy, eco­nomics, law, medicine, and busi­ness.
Jonathan Baron is Pro­fes­sor of Psy­chol­ogy at the Univer­sity of Penn­syl­va­nia. He is the au­thor and ed­i­tor of sev­eral other books, most re­cently Against Bioethics. Cur­rently he is ed­i­tor of the jour­nal Judg­ment and De­ci­sion Mak­ing and pres­i­dent of the So­ciety for Judg­ment and De­ci­sion Mak­ing (2007) .

1. What is think­ing?

This chap­ter will be mostly fa­mil­iar to read­ers of Less Wrong; in the sec­ond para­graph, Baron says (in more words) ‘ra­tio­nal­ity is what wins.’ It may still be helpful as Baron ex­presses a num­ber of things of­ten left un­said here.

He splits think­ing into three parts: think­ing about de­ci­sions (in­stru­men­tal ra­tio­nal­ity), think­ing about be­liefs (epistemic ra­tio­nal­ity), and think­ing about goals. The last is a no­to­ri­ously sticky sub­ject. He also dis­cusses his search-in­fer­ence frame­work, which is how he de­scribes minds as ac­tu­ally op­er­at­ing- com­ing across ideas, eval­u­at­ing them, and pro­ceed­ing from there. Most de­ci­sion anal­y­sis views it­self as op­er­at­ing over a fixed set with a well-defined ob­jec­tive func­tion, but those are the two main prob­lems for real de­ci­sion-mak­ers: iden­ti­fy­ing pos­si­bil­ities worth con­sid­er­ing and com­par­ing two dis­similar out­comes.

The chap­ter is filled out with a dis­cus­sion of un­der­stand­ing, knowl­edge as de­sign, and ex­am­ples of think­ing pro­cesses (worth skim­ming over, but many of which will be fa­mil­iar to ex­perts in the rele­vant fields).

2. The study of thinking

Kah­ne­man and Tver­sky get their first of many refer­ences here. Baron dis­cusses a num­ber of the meth­ods used to learn about hu­man cog­ni­tion, men­tion­ing a few of their pit­falls.

One, which bears re­peat­ing, is that most study of bi­ases just re­ports means, rather than dis­tri­bu­tions. I re­mem­ber learn­ing the ac­tual nu­mer­i­cal size of the Asch con­for­mity ex­per­i­ments about five years af­ter I heard about the ex­per­i­ment it­self, and was un­der­whelmed (32% in­cor­rect an­swers, ~75% of sub­jects gave at least one in­cor­rect an­swer). A gen­eral hu­man ten­dency is differ­ent from a size­able sub­set of weak-willed peo­ple. Similarly, our ar­ti­cle on Prospect The­ory had a link to graphs of sub­jec­tive prob­a­bil­ity in one of the com­ments, of which the most note­wor­thy were the two peo­ple who were nearly lin­ear. While Baron brings up this is­sue, he doesn’t give many ex­am­ples of it here.

He also men­tions three mod­els of thought: de­scrip­tive mod­els, pre­scrip­tive mod­els, and nor­ma­tive mod­els. De­scrip­tive mod­els are what peo­ple ac­tu­ally do; nor­ma­tive mod­els are what thinkers should do with in­finite cog­ni­tive re­sources; pre­scrip­tive mod­els are what thinkers should do with limited cog­ni­tive re­sources. This has come up on LW be­fore, though the fo­cus here has of­ten been ex­clu­sively on the nor­ma­tive, though the pre­scrip­tive seems most use­ful.

Com­puter mod­els of think­ing are briefly dis­cussed, but at a su­perfi­cial level.

This chap­ter sees the first set of ex­er­cises. Over­all, the ex­er­cises in the book seem to provide a brief ex­am­ple /​ check, rather than be­ing enough to de­velop mas­tery. I think this is what I’d recom­mend but it has the po­ten­tial to be a weak­ness.

3. Rationality

Again, Baron iden­ti­fies ra­tio­nal­ity as “the kind of think­ing that helps us achieve our goals.” Refresh­ingly, he fo­cuses on op­ti­mal search, keep­ing in mind the costs of de­ci­sion-mak­ing and in­for­ma­tion-gath­er­ing.

Much of this chap­ter will be fa­mil­iar to some­one who has read the Se­quences, but it’s pre­sented tersely and lu­cidly. The sec­tion on ra­tio­nal­ity and emo­tion, for ex­am­ple, is only three pages long but is clear, quickly iden­ti­fy­ing how the two in­ter­act in a way that’ll clear up com­mon con­fu­sions.

4. Logic

The con­tent in this chap­ter seems mostly unim­por­tant- I imag­ine most read­ers of LW are much more in­ter­ested in prob­a­bil­is­tic rea­son­ing than syl­l­o­gisms. Still, Baron gives a read­able (and not very fa­vor­able) de­scrip­tion of the use­ful­ness of for­mal logic as a nor­ma­tive model of think­ing.

What is fas­ci­nat­ing, though, is the sec­tion of the chap­ter that delves into the four-card prob­lem and vari­a­tions of it. Par­tic­u­larly note­wor­thy is the vari­a­tion de­signed so that most peo­ple’s in­tu­itions are cor­rect- peo­ple give the cor­rect ex­pla­na­tions of why they se­lected the cards they se­lected, and why they didn’t se­lect the cards they didn’t se­lect. But when their in­tu­ition is wrong, they give ex­pla­na­tions that are just as so­phis­ti­cated- but wrong. It’s more ev­i­dence that the de­ci­sion-mak­ing and ver­bal rea­son-pro­vid­ing mod­ules are differ­ent- even some­one who gives the cor­rect ex­pla­na­tion of the cor­rect an­swer may stum­ble on a prob­lem where their un­der­ly­ing sim­ple heuris­tic (pick the cards men­tioned in the ques­tion) fails.

He pre­sents a method of men­tal mod­el­ing that makes log­i­cal state­ments eas­ier to cor­rectly eval­u­ate, and then there are a few log­i­cal in­fer­ence ex­er­cises.

5. Nor­ma­tive the­ory of probability

Yet an­other in­tro­duc­tion to Bayes. Baron fo­cuses pri­mar­ily on Bayesi­anism (called the “per­sonal” the­ory of prob­a­bil­ity) but still in­tro­duces al­ter­na­tives (the “fre­quency” the­ory, i.e. fre­quen­tism, and “log­i­cal” the­ory, which is a sub­set of fre­quen­tism where all events are re­quired to have the same prob­a­bil­ity.) This chap­ter will be use­ful for some­one who doesn’t have a firm prob­a­bil­is­tic foun­da­tion, but holds lit­tle in­ter­est for oth­ers.

There are a hand­ful of ex­er­cises for ap­ply­ing Bayes.

6. De­scrip­tive the­ory of prob­a­bil­ity judgment

This chap­ter pri­mar­ily cov­ers bi­ases re­lated to nu­mer­i­cal prob­a­bil­ity es­ti­mates, many of which are clas­sics in the heuris­tics and bi­ases field (and so have prob­a­bly been men­tioned on Less Wrong at least once). The chap­ter shines when Baron goes into the de­tail of an ex­per­i­ment and its vari­a­tions, as that gives a firmer view of what the ex­per­i­ment ac­tu­ally shows (and, im­por­tantly, what it does not show)- de­scrip­tions of bi­ases where he only quotes a sin­gle ex­per­i­ment (or sin­gle fea­ture of an ex­per­i­ment) feel weaker.

A ma­jor fea­ture of this chap­ter is the im­pli­ca­tion that peo­ple are bad at nu­mer­i­cal prob­a­bil­ity es­ti­ma­tion mostly be­cause they’re un­fa­mil­iar with it, im­ply­ing that cal­ibra­tion ex­er­cises may im­prove prob­a­bil­ity es­ti­ma­tion. A 1977 study of weath­er­man cal­ibra­tion sug­gested they were very well cal­ibrated, both with their es­ti­mates and with the con­fi­dence that should be placed in those es­ti­mates. More re­cent work shows that weath­er­men have sys­tem­atic cal­ibra­tion bi­ases.

7. Hy­poth­e­sis testing

I was grat­ified to dis­cover that this chap­ter was not about statis­tics, but how to come up with and test hy­pothe­ses. Baron dis­cusses differ­ent mod­els of sci­en­tific ad­vance­ment, fo­cus­ing on the sorts of like­li­hood ra­tios that they look for, as well as dis­cussing the sort of mis­takes peo­ple make when choos­ing tests for hy­pothe­ses. Many of the sto­ries will prob­a­bly be fa­mil­iar- Ig­naz Sem­melweis gets a men­tion, though in more de­tail than I had seen be­fore, as well as the 2-4-6 rule fa­mil­iar to HPMOR fans and a vari­a­tion of the four card ex­per­i­ment that makes the typ­i­cal mis­take more ob­vi­ous.

He gives a bak­ing ex­am­ple to sug­gest why peo­ple might search pri­mar­ily for pos­i­tive ev­i­dence- there may be benefits to get­ting a “yes” an­swer be­sides the in­for­ma­tion in­volved. If you’re ex­per­i­ment­ing with cake recipes, and you think your last cake was good be­cause of a fea­ture, it makes sense to al­ter other fea­tures but keep the one you sus­pect the same, as that means a good cake is more likely; if you think a cake was bad be­cause of a fea­ture, it makes sense to al­ter that fea­ture but keep the oth­ers the same, as that also means a good cake is more likely. In a purely sci­en­tific con­text, it makes sense to vary the el­e­ment you think has an im­pact just to max­i­mize the ex­pected size of the im­pact, pos­i­tive or nega­tive.

He de­scribes in more de­tail a method­ol­ogy he’s been dis­cussing, “ac­tively open-minded think­ing,” which seems to boil down to “don’t just be will­ing to ac­cept dis­con­firm­ing ev­i­dence, go look­ing for it,” but the full ex­pla­na­tion comes in a few chap­ters.

8. Judg­ment of cor­re­la­tion and contingency

This chap­ter is de­scrip­tive; it be­gins with a de­scrip­tion of cor­re­la­tions and then dis­cusses hu­man judg­ment of cor­re­la­tions. Un­sur­pris­ingly, peo­ple suffer from the illu­sion of con­trol- they think there’s more likely to be a cor­re­la­tion if their effort is in­volved- and from con­fir­ma­tion bias. There are some ex­am­ples of the lat­ter, where peo­ple find cor­re­la­tions that make in­tu­itive sense but aren’t in the data, and don’t dis­cover cor­re­la­tions that don’t make in­tu­itive sense that are in the data. There’s also a brief sec­tion on how peo­ple use nearly use­less ev­i­dence to sup­port the­o­ries or dis­miss ev­i­dence that doesn’t sup­port their the­ory. Over­all, it’s a short chap­ter that won’t be sur­pris­ing to LW read­ers (al­though some of the stud­ies refer­enced may be new).

9. Ac­tively open-minded thinking

I’ll quote part of this chap­ter in full be­cause I think it’s a great de­scrip­tion:

[G]ood think­ing con­sists of (1) search that is thor­ough in pro­por­tion to the im­por­tance of the ques­tion, (2) con­fi­dence that is ap­pro­pri­ate to the amount and qual­ity of think­ing done, and (3) fair­ness to other pos­si­bil­ities than the one we ini­tially fa­vor.

The chap­ter over­all is very solid- it deftly com­bines nor­ma­tive pre­dic­tions with de­scrip­tive bi­ases to weave a pre­scrip­tive recom­men­da­tion of how to think bet­ter. There are sev­eral great ex­am­ples of ac­tively open-minded think­ing; in par­tic­u­lar, the thought pro­cess of two stu­dents as they at­tempt to make sense of a story sen­tence by sen­tence.

Many of the sug­ges­tions in the chap­ter are ex­tended by var­i­ous LW posts, but the chap­ter seems use­ful as a con­cise de­scrip­tion of the whole prob­lem and illus­tra­tion of a gen­eral solu­tion. If you’re hav­ing trou­ble fit­ting to­gether var­i­ous ra­tio­nal­ity hacks, this seems like a good ban­ner to unite them un­der.

10. Nor­ma­tive the­ory of choice un­der uncertainty

This chap­ter is an in­tro­duc­tion to util­ity the­ory, de­scribing how it works, how mul­ti­ple at­tributes can be con­soli­dated into one score, and a way to re­solve con­flicts be­tween agents with differ­ent util­ities. It’s a good in­tro­duc­tion to de­ci­sion anal­y­sis /​ util­ity the­ory, and there are some ex­er­cises, but there are no sur­prises for some­one who’s seen this be­fore.

11. De­scrip­tive the­ory of choice un­der uncertainty

This chap­ter is an in­tro­duc­tion to differ­ent the­o­ries of how hu­mans ac­tu­ally make de­ci­sions, like prospect the­ory and re­gret the­ory. There are a hand­ful of ex­er­cises for un­der­stand­ing prospect the­ory.

Baron takes an even-handed ap­proach to de­vi­a­tions from the nor­ma­tive the­ory. For ex­am­ple, when dis­cussing re­gret the­ory, re­grets have a real emo­tional cost (and real learn­ing benefit)- but be­hav­ing ac­cord­ing to de­scrip­tive the­o­ries be­cause they’re de­scrip­tive rather than be­cause they’re use­ful is a mis­take. In many cases, those emo­tions can be ma­nipu­lated by choice of refer­ence point.

He also dis­cusses the am­bi­guity effect- where peo­ple treat known prob­a­bil­ities differ­ently from un­known prob­a­bil­ities, giv­ing ex­am­ples both of lab­o­ra­tory situ­a­tions (draw­ing balls from an urn with a par­tially known com­po­si­tion) and real-life situ­a­tions (in­sur­ing un­prece­dented or un­re­peat­able events). Baron de­scribes this as in­com­pat­i­ble with per­sonal prob­a­bil­ity and sug­gests it’s re­lated to fram­ing- situ­a­tions where the prob­a­bil­ities seem known can be changed into situ­a­tions where prob­a­bil­ities seem un­known. This aver­sion to am­bi­guity, though, can be perfectly sen­si­ble in­so­far as it pushes de­ci­sion-mak­ers to ac­quire more in­for­ma­tion.

He also dis­cusses a Tver­sky study in which most stu­dents make a de­ci­sion to pay money to defer a de­ci­sion un­til they re­ceive rele­vant in­for­ma­tion, but when asked how they would make the de­ci­sion in the case of ei­ther pos­si­ble piece of in­for­ma­tion, most stu­dents re­al­ize they would make the same de­ci­sion and choose not to defer the de­ci­sion.

12. Choice un­der certainty

This chap­ter is pri­mar­ily de­scrip­tive, fo­cus­ing on the prob­lem of think­ing about goals. Most peo­ple fa­vor cat­e­gor­i­cal goal sys­tems- Baron gives a great ex­am­ple, from Gar­diner and Ed­wards, of the Cal­ifor­nia Coastal Com­mis­sion, tasked to de­cide which de­vel­op­ment pro­jects to al­low on the Pa­cific Coast. The com­mis­sion was split into pro-de­vel­op­ment and pro-en­vi­ron­ment fac­tions, which al­most never agreed on which pro­jects to al­low and dis­al­low. When asked to rank pro­jects, most would rank them solely by their preferred crite­rion, cre­at­ing lists that strongly dis­agreed. When asked to take both crite­ria into ac­count- but with what­ever weight­ing they wanted- the sub­jects would heav­ily weight their preferred crite­rion, but the pro­jects which were both very valuable and not very en­vi­ron­men­tally dam­ag­ing floated to the top of both lists, cre­at­ing sig­nifi­cant agree­ment.

The list of bi­ases is long, and each has a study or story as­so­ci­ated with. Many of the effects have been men­tioned on LW some­where, but it’s very use­ful to have them placed next to each other (and sep­a­rated from prob­a­bil­is­tic bi­ases), and so I’d recom­mend ev­ery­one read this chap­ter.

13. Utility measurement

This de­scrip­tive chap­ter dis­cusses the difficult challenge of mea­sur­ing util­ities. It in­tro­duces both de­ci­sion anal­y­sis and cost-benefit anal­y­sis- the lat­ter con­verts out­comes to dol­lars to guide de­ci­sions, while the former con­verts out­comes to util­ity val­ues to guide de­ci­sions.

Peo­ple are not very skil­led at satis­fy­ing ax­ioms we would like them to satisfy. For ex­am­ple, con­sider the challenge of valu­ing a cer­tain $50 against a p chance of $100 (and $0 oth­er­wise). A sub­ject will of­ten give an an­swer like .7. Then, when later asked how much a 70% chance of $100 is worth, the sub­ject will an­swer $60. That in­con­sis­tency needs to be re­solved be­fore their an­swers are used as pa­ram­e­ters for any de­ci­sions. Thank­fully, this is an area of ac­tive re­search, and ways to elicit prob­a­bil­ities and val­ues that hold up to re­flec­tive equil­ibrium are grad­u­ally be­ing de­vel­oped. (This par­tic­u­lar chap­ter, while it sounds that note of hope, is mostly nega­tive: here are meth­ods that have been tried and have crip­pling prob­lems.)

This seems like a chap­ter that would be use­ful for any­one who wants to use util­ities in an ar­gu­ment or model- treat­ing them like they’re un­am­bigu­ous, eas­ily mea­sured ob­jects when they ac­tu­ally seem to be fuzzy and hard to pin down can lead to sig­nifi­cant prob­lems, and think­ing clearly about val­ues is a spot where LW could do bet­ter.

14. De­ci­sion anal­y­sis and values

This chap­ter is a more pre­scrip­tive ap­proach to the same prob­lem- given that util­ities and val­ues are hard to find, where do we look for them? A di­chotomy fa­mil­iar to LW read­ers- in­stru­men­tal and ter­mi­nal val­ues- ap­pears here as “means-ends ob­jec­tive hi­er­ar­chy” or “means val­ues” and “fun­da­men­tal val­ues.”

It con­tains a wealth of ex­am­ples, in­clud­ing a com­puter-buy­ing one with po­ten­tial mem­o­ries of 64KB to 640KB, with the hilar­i­ous com­ment that “you are buy­ing this com­puter many years ago, when these num­bers made sense!” There are also prac­ti­cal elic­i­ta­tion sug­ges­tions- rather than try to figure out a point es­ti­mate, start from a num­ber that’s too high un­til you’re in­differ­ent, and then start from a num­ber that’s too low un­til you’re in­differ­ent, giv­ing you an in­differ­ence range (that you can ei­ther re­port or use the mid­dle of as a point es­ti­mate).

Lex­i­cal prefer­ences (also called cat­e­gor­i­cal prefer­ences el­se­where) and trade­offs are dis­cussed- Baron takes the po­si­tion (that I share) that lex­i­cal prefer­ences are ac­tu­ally trade­offs with very, very high weights. (How do we trade off hu­man lives and dol­lars? We should re­quire a lot of dol­lars for a life- but not an in­finite amount.) There’s a dis­cus­sion of micro­morts (though he doesn’t use that term) and of his­tor­i­cal at­tempts to teach de­ci­sion anal­y­sis that should be in­ter­est­ing to CFAR (though the refer­ences are a few decades old, now). The dis­cus­sion of the ex­am­ples con­tains quite a bit of prac­ti­cal ad­vice, and the chap­ter seems worth­while for al­most ev­ery­one.

15. Quan­ti­ta­tive judgment

This chap­ter de­scribes three com­mon quan­ti­ta­tive prob­lems- scor­ing, rank­ing, and clas­sify­ing, and dis­cusses some bi­ases that ham­per hu­man de­ci­sion-mak­ing along those lines and some recom­men­da­tions. Statis­ti­cal pre­dic­tion rules make an ap­pear­ance, though they’re not called that. One fas­ci­nat­ing sug­ges­tion is that mod­els of peo­ple can ac­tu­ally perform bet­ter than those peo­ple, since the mod­els don’t have off days and peo­ple do.

This chap­ter will have some new ma­te­rial for LWers, and seems like a good ex­ten­sion of the pre­vi­ous chap­ter.

16. Mo­ral Judg­ment and Choice

This chap­ter dis­cusses moral­ity from the point of de­ci­sion-mak­ing- which is a re­fresh­ing per­spec­tive. Baron strongly en­dorses con­se­quen­tial­ism and weakly en­dorses util­i­tar­i­anism, pro­vid­ing a host of moral ques­tions in which many peo­ple de­vi­ate from the con­se­quen­tial­ist or util­i­tar­ian po­si­tion.

A re­cur­ring theme is omis­sion bias: peo­ple tend to judge ac­tive in­volve­ment in a situ­a­tion in which some­one is made worse off as worse than pas­sive in­volve­ment in such a situ­a­tion, even if the end re­sult is bet­ter for ev­ery­one. Peo­ple also weight in­ten­tions, which doesn’t fit a di­rect con­se­quen­tial­ist view.

Over­all, the chap­ter seems valuable for re­fram­ing moral ques­tions- plac­ing them within the realm of prag­ma­tism by mov­ing to the per­spec­tive of de­ci­sions- but pro­vides very lit­tle in the way of an­swers. Both the con­se­quen­tial­ist and util­i­tar­ian po­si­tions are con­tro­ver­sial and come with sig­nifi­cant draw­backs, and Baron is fair enough in pre­sent­ing those draw­backs and con­tro­ver­sies, though in a rather abridged form.

17. Fair­ness and justice

This chap­ter is an ex­ten­sion of the pre­vi­ous chap­ter, fo­cus­ing on in­tu­itions deal­ing with fair­ness and jus­tice. Baron de­tails situ­a­tions in which they agree and dis­agree with util­i­tar­ian anal­y­sis. Note­wor­thy is the un­der­cur­rent of adap­ta­tion-ex­e­cu­tion and not util­ity-max­i­miza­tion—fair­ness has tan­gible benefits, but peo­ple will of­ten pur­sue fair­ness even at the cost of tan­gible benefits.

This chap­ter (and to a lesser ex­tent the pre­vi­ous one) seem odd in light of chap­ter 15, in which the fal­li­bil­ity of in­di­vi­d­ual judg­ment took cen­ter stage, with the recom­men­da­tion that ap­ply­ing rules de­rived from in­di­vi­d­ual judg­ment can of­ten do bet­ter. It is good to know the rea­son­ing that jus­tifies moral in­tu­itions, es­pe­cially if one is in­ter­ested in their bound­aries, but when those bound­aries im­pact out­comes they be­come poli­ti­cal ques­tions. If the sole point of pun­ish­ment is de­ter­rence (and that is the only sen­si­ble util­i­tar­ian jus­tifi­ca­tion), the ques­tion of whether or not a de­ci­sion can im­pact fu­ture de­ci­sions is a sticky one. Per­haps the full con­se­quen­tial­ist reck­on­ing will recom­mend un­think­ing ap­pli­ca­tion of the rules, even in cases where di­rect con­se­quen­tial­ist reck­on­ing recom­mends sus­pend­ing the rules.

18. So­cial dilem­mas: co­op­er­a­tion ver­sus defection

This chap­ter fo­cuses on de­scrip­tive ex­per­i­ments- how peo­ple ac­tu­ally be­have in so­cial dilem­mas- find­ing them to be much more co­op­er­a­tive than nor­ma­tive the­ory would recom­mend. There is some am­bi­guity, which he dis­cusses, in what the “nor­ma­tive the­ory” is- util­i­tar­i­anism recom­mends co­op­er­a­tion on the pris­oner’s dilemma, for ex­am­ple, be­cause it max­i­mizes to­tal util­ity, whereas ex­pected util­ity the­ory recom­mends defec­tion on the pris­oner’s dilemma, be­cause it’s a dom­i­nat­ing strat­egy.

The value of the chap­ter mostly lies in the study re­sults- a few are in­ter­est­ing, like that dis­cussing the so­cial dilemma with other par­ti­ci­pants be­fore­hand sig­nifi­cantly in­creases co­op­er­a­tion, or that sub­jects are more likely to defect on the pris­oner’s dilemma if they know their part­ner’s re­sponse than if they are un­cer­tain, even if they know their part­ner co­op­er­ated.

Typ­i­cally, for so­cial dilem­mas (sce­nar­ios in which pri­vate gain re­quires pub­lic loss, or pub­lic gain re­quires pri­vate loss), de­ci­sion-mak­ing bi­ases in­crease the level that peo­ple co­op­er­ate. (This is some­what un­sur­pris­ing, since the nor­ma­tive recom­men­da­tion is typ­i­cally defec­tion, and bi­ases move real de­ci­sions away from the nor­ma­tive recom­men­da­tion.) Peo­ple fail to dis­t­in­guish be­tween ca­sual in­fluence- “my vot­ing makes peo­ple like me more likely to vote”- from di­ag­nos­tic in­fluence- “peo­ple like me vot­ing makes me more likely to vote”- but one of the ma­jor rea­sons peo­ple give for vot­ing is that it has a causal in­fluence, rather than a merely di­ag­nos­tic one.

19. De­ci­sions about the future

This chap­ter is un­likely to con­tain any sur­prises for LWers, but serves as a fine in­tro­duc­tion to dis­count­ing, both ex­po­nen­tial and hy­per­bolic, and thus dy­namic in­con­sis­tency. Also in­ter­est­ing (but too brief) is the dis­cus­sion of goals in the con­text of time and plans and of goals as malle­able ob­jects.

Baron de­scribes four meth­ods of self-con­trol: ex­trapsy­chic de­vices (re­mov­ing a tempt­ing op­tion), con­trol of at­ten­tion (think­ing about things other than the tempt­ing op­tion), con­trol of emo­tion (cul­ti­vat­ing an in­com­pat­i­ble emo­tion), or per­sonal rules (view­ing situ­a­tions as in­stances of gen­eral poli­cies, rather than iso­lated events). Again, the dis­cus­sion is brief- only two pages- though the sub­ject is of great in­ter­est to many here.

20. Risk

This chap­ter fo­cuses on de­scrip­tive ap­proaches to risk- sur­vey re­sponses and gov­ern­ment reg­u­la­tion- as the nor­ma­tive ap­proach to risk has mostly been de­tailed in the rest of the book: use ex­pected util­ity the­ory. Most peo­ple are be­set by bi­ases and in­nu­mer­acy, though, and so there’s a whole chap­ter of ma­te­rial on mis­judg­ments of risk and in­surance.

Many of the bi­ases, though per­haps not the ex­am­ples, will be fa­mil­iar to LWers. On the whole, they’re some­what un­in­ter­est­ing since most of them seem to just re­sult from in­nu­mer­acy: when given a table of deaths per year from four causes with wildly differ­ent prevalences, sub­jects were cor­rectly will­ing to pay more to re­duce larger risks by the same per­centage as smaller risks. But their prefer­ences scaled much more slowly than the risks- the sub­jects were, on av­er­age, will­ing to pay 20 times as much to pre­vent 20% of the deaths from a cause of death that kil­led 10,000 times as many peo­ple. Those dis­torted will­ing­nesses to pay show up in gov­ern­ment reg­u­la­tions. Peo­ple were also more will­ing to pay for pro­tec­tion against the un­fa­mil­iar than the fa­mil­iar- even though the rel­a­tive benefit was far higher for pro­tec­tion against the fa­mil­iar. (The illu­sion of con­trol also shows up, dis­tort­ing per­cep­tions of risk.)


What should I read?

  • Al­most ev­ery­one: 7 and 9.

  • I’m hunt­ing bi­ases: 6, 8, 11, 12, and then 15-20 (per­haps with­out 18).

  • I’m in­ter­ested in moral rea­son­ing: 13 and 16 should be re­quired read­ing. 14, 15, and 17-19 will be use­ful.

  • I’m a de­ci­sion maker: 10 and 14 will be di­rectly use­ful, but check out the bias chap­ters too.

  • I’m new to ra­tio­nal­ity: Start off with 1-4.

  • I’m an ex­pert at ra­tio­nal­ity but haven’t heard of Baron: Still read 1-4, just to get his per­spec­tive of the field.

  • I don’t have a strong back­ground in Bayesi­anism: read chap­ter 5.