The Truth and Instrumental Rationality

One of the cen­tral fo­cuses of LW is in­stru­men­tal ra­tio­nal­ity. It’s been sug­gested, rather fa­mously, that this isn’t about hav­ing true be­liefs, but rather its about “win­ning”. Sys­tem­atized win­ning. True be­liefs are of­ten use­ful to this goal, but an ob­ses­sion with “truthi­ness” is seen as counter-pro­duc­tive. The brilli­ant sci­en­tist or philoso­pher may know the truth, yet be in­effec­tive. This is seen as un­ac­cept­able to many who see in­stru­men­tal ra­tio­nal­ity as the crit­i­cal path to achiev­ing one’s goals. Should we all dis­card our philo­soph­i­cal ob­ses­sion with the truth and be­come “win­ners”?

The River In­stru­men­tus

You are lead­ing a group of five peo­ple away from deadly threat which is slowly ad­vanc­ing be­hind you. You come to a river. It looks too dan­ger­ous to wade through, but through the spray of the wa­ter you see a num­ber of stones. They are dot­ted across the river in a way that might al­low you to cross. How­ever, the five peo­ple you are helping are ex­tremely ner­vous and in or­der to con­vince them to cross, you will not only have to show them its pos­si­ble to cross, you will also need to look calm enough af­ter do­ing it to con­vince them that it’s safe. All five of them must cross, as they in­sist on liv­ing or dy­ing to­gether.

Just as you are about to step out onto the first stone it splut­ters and moves in the mist of the spray­ing wa­ter. It looks a lit­tle differ­ent from the oth­ers, now you think about it. After a mo­ment you re­al­ise its ac­tu­ally a per­son, strug­gling to keep their head above wa­ter. Your best guess is that this per­son would prob­a­bly drown if they got stepped on by five more peo­ple. You think for a mo­ment, and de­cide that, be­ing a con­se­quen­tial­ist con­cerned pri­mar­ily with the preser­va­tion of life, it is ul­ti­mately bet­ter that this per­son dies so the oth­ers wait­ing to cross might live. After all, what is one life com­pared with five?

How­ever, given your need for calm and the hor­ror of their im­mi­nent death at your hands (or feet), you de­cide it is bet­ter not to think of them as a per­son, and so you in­stead imag­ine them be­ing sim­ply a stone. You know you’ll have to be re­ally con­vinc­ingly calm about this, so you look at the top of the head for a full hour un­til you ut­terly con­vince your­self that the shape you see be­fore you is fac­tu­ally in­dic­i­ta­tive not of a per­son, but of a stone. In your mind, tops of heads aren’t peo­ple—now they’re stones. This is in­stru­men­tally ra­tio­nal—when you weigh things up the self-de­cep­tion ul­ti­mately in­creases the num­ber of peo­ple who will likely live, and there is no spe­cific harm you can iden­tify as a re­sult.

After you have finished con­vinc­ing your­self you step out onto the per… stone… and start cross­ing. How­ever, as you step out onto the sub­se­quent stones, you no­tice they all shift a lit­tle un­der your feet. You look down and see the stones splut­ter­ing and strug­gling. You think to your­self “lucky those stones are stones and not peo­ple, oth­er­wise I’d be re­ally up­set”. You lead the five very great­ful peo­ple over the stones and across the river. Twenty dead stones drift silently down­stream.

When we weigh situ­a­tions on pure in­stru­men­tal­ity, small self de­cep­tion makes sense. The only prob­lem is, in an am­bigu­ous and com­plex world, self-de­cep­tions have a no­to­ri­ous way of com­pound­ing ea­chother, and leave a gap­ing hole for cog­ni­tive bias to work its magic. Many false but deeply-held be­liefs through­out hu­man his­tory have been quite jus­tifi­able on these grounds. Yet when we for­get the value of truth, we can be in­stru­men­tal, but we are not in­stru­men­tally ra­tio­nal. Ra­tion­al­ity im­plies, or ought to im­ply, a value of the truth.

Win­ning and sur­vival

In the jun­gle of our evolu­tion­ary child­hood, hu­man­ity formed groups to sur­vive. In these groups there was a hi­er­achy of im­por­tance, sta­tus and power. Preda­tors, star­va­tion, ri­val groups and dis­ease all took the weak on a reg­u­lar ba­sis, but the groups af­forded a par­tial pro­tec­tion. How­ever, a vi­o­lent or un­pleas­ant death still re­mained a con­stant threat. It was of par­tic­u­lar threat to the low­est and weak­est mem­bers of the group. Some­times these in­di­vi­d­u­als were weak be­cause they were phys­i­cally weak. How­ever, over time groups that al­lowed and re­warded things other than phys­i­cal strength be­came more suc­cess­ful. In these groups, dis­cus­sion played a much greater role in power and sta­tus. The tru­ely strong in­di­vi­d­u­als, the win­ners in this new arena were one’s that could di­rect con­ver­sta­tion in their favour—con­ver­sa­tions about who will do what, about who got what, and about who would be pun­ished for what. De­bates were fought with words, but they could end in death all the same.

In this en­vi­ron­ment, one’s so­cial sta­tus is in­ter­twined with one’s abil­ity to win. In a de­bate, it was not so much a mat­ter of what was true, but of what facts and be­liefs achieved one’s goals. Sup­port­ing the fac­tual po­si­tion that suited one’s own goals was most im­por­tant. Even where the stakes where low or ir­rele­vant, it payed to pre­vail so­cially, be­cause one’s rep­u­ta­tion guided oth­ers limited cog­ni­tion about who was best to listen to. Win­ning didn’t mean know­ing the most, it meant so­cial vic­tory. So when com­pe­ti­tion bub­bled to the sur­face, it payed to ig­nore what one’s op­po­nent said and in­stead fo­cus on ap­pear­ing su­pe­rior in any way pos­si­ble. Sure, truth some­times helped, but for the charis­matic it was strictly op­tional. Poli­tics was born.

Yet as groups got larger, and as tech­nol­ogy be­gan to ad­vance for the first time, there ap­peared a new phe­nomenon. Where a group’s power dy­nam­ics meant that it sys­tem­at­i­cally had false be­liefs, it be­came more likely to fail. The group that be­liev­ing that fire spirits guided a fire’s ad­vance­ment fared poorly com­pared with those who checked the wind and planned their means of es­cape ac­cord­ingly. The truth fi­nally came into its own. Yet truth, as op­posed to sim­ple be­lief by poli­tics, could not be so eas­ily ma­nipu­lated for per­sonal gain. The truth had no mas­ter. In this way it was both dan­ger­ous and liber­at­ing. And so slowly but surely the ca­pac­ity for com­plex truth-pur­suit be­came evolu­tion­ar­ily im­pressed upon the hu­man blueprint.

How­ever, in evolu­tion­ary terms there was lit­tle time for the com­ple­tion of this new men­tal state. Some peo­ple had it more than oth­ers. It also re­quired the right cir­cum­stances for it to rise to the fore­front of hu­man thought. And other con­di­tions could eas­ily de­stroy it. For ex­am­ple, should a per­son’s thoughts be primed with an en­vi­ron­ment of com­pe­ti­tion, the old ways came bub­bling up to the sur­face. When a per­son’s en­vi­ron­ment is highly com­pet­i­tive, it re­verts to its prim­i­tive state. Learn­ing and up­dat­ing of views be­comes in­creas­ingly difficult, be­cause to the more prim­i­tive as­pects of a per­son’s so­cial brain, up­dat­ing one’s views is a so­cial defeat.

When we fo­cus an or­gani­sa­tion’s cul­ture on win­ning, there can be many benefits. It can cre­ate an air of achieve­ment, to a de­gree. Hard work and the challeng­ing of norms can be in­creased. How­ever, we also prime the brain for so­cial con­flict. We cre­ate an en­vi­ron­ment where com­plex­ity and sub­tlety in con­ver­sa­tion, and con­se­quently in thought, is greatly re­duced. In or­gani­sa­tions where the goals and means are largely in­tel­lec­tual, a com­pet­i­tive en­vi­ron­ment cre­ates use­less con­ver­sa­tions, mean­ingless de­bates, pointless trib­al­ism, and lit­tle mean­ingful learn­ing. There are many great ex­am­ples, but I think you’d be best served watch­ing our elected rep­re­sen­ta­tives at work to gain a real in­sight.

Ra­tion­al­ity and truth

Ra­tion­al­ity ought to con­tain an im­pli­ca­tion of truth­ful­ness. Without it, our lit­tle self-de­cep­tions start to gather and com­pond one an­other. Slowly but surely, they start to re­in­force, join, and form an un­break­able, un­challengable yet ut­terly false be­lief sys­tem. I need not point out the more ob­vi­ous ex­am­ples, for in hu­man so­ciety, there are many. To avoid this on LW and el­se­where, truth­ful­ness of be­lief ought to in­form all our ra­tio­nal de­ci­sions, meth­ods and goals. Of course true be­liefs do not guaran­tee in­fluence or power or achieve­ment, or any­thing re­ally. In a world of half-evolved truth-seek­ing equip­ment, why would we ex­pect that? What we can ex­pect is that, if our goals are any­thing to do with the mod­ern world in all its com­plex­ity, the truth isn’t suffi­cient, but it is nec­ces­sary.

In­stru­men­tal ra­tio­nal­ity is about achiev­ing one’s goals, but in our com­plex world goals man­i­fest in many ways—and we can never re­ally pre­dict how a false be­lief will dis­tort our ac­tions to ut­terly de­stroy our ac­tual achieve­ments. In the end, with­out truth, we never re­ally see the stones float­ing down the river for what they are.