What are the optimal biases to overcome?

If you’re in­ter­ested in learn­ing ra­tio­nal­ity, where should you start? Re­mem­ber, in­stru­men­tal ra­tio­nal­ity is about mak­ing de­ci­sions that get you what you want—surely there are some les­sons that will help you more than oth­ers.

You might start with the most fa­mous ones, which tend to be the ones pop­u­larized by Kah­ne­man and Tver­sky. But K&T were aca­demics. They weren’t try­ing to help peo­ple be more ra­tio­nal, they were try­ing to prove to other aca­demics that peo­ple were ir­ra­tional. The re­sult is that they fo­cused not on the most im­por­tant bi­ases, but the ones that were eas­iest to prove.

Take their fa­mous an­chor­ing ex­per­i­ment, in which they showed the spin of a roulette wheel af­fected peo­ple’s es­ti­mates about Afri­can coun­tries. The idea wasn’t that roulette wheels caus­ing bi­ased es­ti­mates was a huge so­cial prob­lem; it was that no aca­demic could pos­si­bly ar­gue that this be­hav­ior was some­how ra­tio­nal. They thereby scored a de­ci­sive blow for psy­chol­ogy against economists claiming we’re just ra­tio­nal max­i­miz­ers.

Most aca­demic work on ir­ra­tional­ity has fol­lowed in K&T’s foot­steps. And, in turn, much of the stuff done by LW and CFAR has fol­lowed in the foot­steps of this aca­demic work. So it’s not hard to be­lieve that LW types are good at avoid­ing these bi­ases and thus do well on the psy­chol­ogy tests for them. (In­deed, many of the ques­tions on these tests for ra­tio­nal­ity come straight from K&T ex­per­i­ments!)

But if you look at the av­er­age per­son and ask why they aren’t get­ting what they want, very rarely do you con­clude their biggest prob­lem is that they’re suffer­ing from an­chor­ing, fram­ing effects, the plan­ning fal­lacy, com­mit­ment bias, or any of the other stuff in the se­quences. Usu­ally their biggest prob­lems are far more quo­tidian and com­mon­sen­si­cal.

Take Eliezer. Surely he wanted SIAI to be a well-func­tion­ing or­ga­ni­za­tion. And he’s ad­mit­ted that luke­prog has done more to achieve that goal of his than he has. Why is luke­prog so much bet­ter at get­ting what Eliezer wants than Eliezer is? It’s surely not be­cause luke­prog is so much bet­ter at avoid­ing Se­quence-style cog­ni­tive bi­ases! luke­prog read­ily ad­mits that he’s con­stantly learn­ing new ra­tio­nal­ity tech­niques from Eliezer.

No, it’s be­cause luke­prog did what seems like com­mon sense: he bought a copy of Non­prof­its for Dum­mies and did what it recom­mends. As luke­prog him­self says, it wasn’t lack of in­tel­li­gence or re­sources or akra­sia that kept Eliezer from do­ing these things, “it was a gap in gen­eral ra­tio­nal­ity.”

So if you’re in­ter­ested in clos­ing the gap, it seems like the skills to pri­ori­tize aren’t things like com­mit­ment effect and the sunk cost fal­lacy, but stuff like “figure out what your goals re­ally are”, “look at your situ­a­tion ob­jec­tively and list the biggest prob­lems”, “when you’re try­ing some­thing new and risky, read the For Dum­mies book about it first”, etc. For lack of bet­ter ter­minol­ogy, let’s call the K&T stuff “cog­ni­tive bi­ases” and this stuff “prac­ti­cal bi­ases” (even though it’s all ob­vi­ously both prac­ti­cal and cog­ni­tive and bi­ases is kind of a nega­tive way of look­ing at it).

What are the best things you’ve found on tack­ling these “prac­ti­cal bi­ases”? Post your sug­ges­tions in the com­ments.