Epistemic Luck

Who we learn from and with can profoundly in­fluence our be­liefs. There’s no ob­vi­ous way to com­pen­sate. Is it time to panic?

Dur­ing one of my episte­mol­ogy classes, my pro­fes­sor ad­mit­ted (I can’t re­call the con­text) that his opinions on the topic would prob­a­bly be differ­ent had he at­tended a differ­ent grad­u­ate school.

What a pe­cu­liar thing for an episte­mol­o­gist to ad­mit!

Of course, on the one hand, he’s al­most cer­tainly right. Schools have their cul­tures, their tra­di­tional views, their fa­vorite liter­a­ture providers, their set of available teach­ers. Th­ese have a de­cided enough effect that I’ve heard “X was a stu­dent of Y” used to mean “X holds views ba­si­cally like Y’s”. And ev­ery­body knows this. And peo­ple still show a dis­tinct trend of agree­ing with their teach­ers’ views, even the most con­tro­ver­sial—not an un­bro­ken trend, but still an ob­vi­ous one. So it’s not at all un­likely that, yes, had the pro­fes­sor gone to a differ­ent grad­u­ate school, he’d be­lieve some­thing else about his sub­ject, and he’s not mak­ing a mis­take in so ac­knowl­edg­ing...

But on the other hand… but… but...

But how can he say that, and look so un­du­bi­ously at the views he picked up this way? Surely the truth about knowl­edge and jus­tifi­ca­tion isn’t cor­re­lated with which school you went to—even a lit­tle bit! Surely he knows that!

And he does—and so do I, and it doesn’t stop it from hap­pen­ing. I even iden­ti­fied a quale as­so­ci­ated with the in­ex­orable slide to­wards a con­sen­sus po­si­tion, which made for some in­ter­est­ing in­tro­spec­tion, but averted no change of mind. Be­cause what are you sup­posed to do—re­s­olutely hold to what­ever in­tu­itions you walked in with, never mind the coax­ing and ar­gu­ing and ever-so-rea­son­able per­sua­sions of the en­vi­ron­ment in which you are steeped? That won’t do, and not only be­cause it ob­vi­ates the ed­u­ca­tion. The truth isn’t an­ti­cor­re­lated with the school you go to, ei­ther!

Even if ev­ery­one col­lec­tively at­tempted this stub­born­ness only to the ex­act de­gree needed to re­move the statis­ti­cal con­nec­tion be­tween teach­ers’ views and their stu­dents’, it’s still not truth-track­ing. An anal­ogy: sup­pose you give a stan­dard­ized English lan­guage test, de­ter­mine that His­pan­ics are do­ing dis­pro­por­tionately well on it, figure out that this is be­cause many speak Ro­mance lan­guages and do well with Lati­nate words, and deflate His­panic scores to even out the de­mo­graph­ics of the test re­sults. This might give you a racially bal­anced out­come, but on an in­di­vi­d­ual level, it will un­fairly hurt some mono­lin­gual An­glo­phone His­pan­ics, and help some Fran­co­phone test-tak­ers—it will not do as much as you’d hope to im­prove the skill-track­ing abil­ity of the test. Similarly, flat­ten­ing the im­pact of teach­ing on stu­dent views won’t sal­vage truth-track­ing of stu­dent views as though this trend never ex­isted; it’ll just yield the same high-level statis­tics you’d get if that bias weren’t op­er­at­ing.

Lots of bi­ases still live in your head do­ing their thing even when you know about them. This one, though, puts you in an awfully weird epistemic situ­a­tion. It’s al­most like the op­po­site of be­lief in be­liefdis­be­lief in be­lief. “This is true, but my situ­a­tion made me more prone than I should have been to be­lieve it and my be­lief is there­fore sus­pect. But dang, that ar­gu­ment my teacher ex­plained to me sure was sound-look­ing! I must just be lucky—those poor saps with other teach­ers have it wrong! But of course I would think that...”

It is pos­si­ble, to an ex­tent, to re­duce the risk here—you can sur­round your­self with cog­ni­tively di­verse peers and teach­ers, even if only in un­offi­cial ca­pac­i­ties. But even then, who you spend the most time with, whom you get along with best, whose style of thought “clicks” most with yours, and—due to com­pet­ing bi­ases—who­ever agrees with you already will have more of an effect than the oth­ers. In prac­tice, you can’t sit your­self in a con­trol­led en­vi­ron­ment and ex­pose your­self to pure and perfect ar­gu­ment and ev­i­dence (with­out al­low­ing ac­ci­den­tal lean­ings to creep in via the or­der in which you read it, ei­ther).

I’m not even sure if it’s right to as­sign a higher con­fi­dence to be­liefs that you hap­pen to have main­tained—ab­sent spe­cial effort—in con­tra­ven­tion of the gen­eral agree­ment. It seems to me that peo­ple have trains of thought that just seem more nat­u­ral to them than oth­ers. (Was I the only one dis­con­certed by Eliezer an­nounc­ing high con­fi­dence in Bayesi­anism in the same post as a state­ment that he was prob­a­bly “born that way”?) This isn’t even a highly re­li­able way for you to learn things about your­self, let alone the rest of the world: un­less there’s a spe­cial rea­son your in­tu­itions—and not those of peo­ple who think differ­ently—should be truth-track­ing, these be­liefs are likely to rep­re­sent where your brain just hap­pens to clamp down re­ally hard on some­thing and re­sist group pres­sure and that in­ex­orable slide.