Fact Posts: How and Why

The most use­ful think­ing skill I’ve taught my­self, which I think should be more widely prac­ticed, is writ­ing what I call “fact posts.” I write a bunch of these on my blog. (I write fact posts about preg­nancy and child­birth here.)

To write a fact post, you start with an em­piri­cal ques­tion, or a gen­eral topic. Some­thing like “How com­mon are hate crimes?” or “Are epi­du­rals re­ally dan­ger­ous?” or “What causes man­u­fac­tur­ing job loss?”

It’s okay if this is a topic you know very lit­tle about. This is an ex­er­cise in origi­nal see­ing and show­ing your rea­son­ing, not find­ing the offi­cial last word on a topic or do­ing the best anal­y­sis in the world.

Then you open up a Google doc and start tak­ing notes.

You look for quan­ti­ta­tive data from con­ven­tion­ally re­li­able sources. CDC data for in­ci­dences of dis­eases and other health risks in the US; WHO data for global health is­sues; Bureau of La­bor Statis­tics data for US em­ploy­ment; and so on. Pub­lished sci­en­tific jour­nal ar­ti­cles, es­pe­cially from rep­utable jour­nals and large ran­dom­ized stud­ies.

You ex­plic­itly do not look for opinion, even ex­pert opinion. You avoid news, and you’re wary of think-tank white pa­pers. You’re look­ing for raw in­for­ma­tion. You are tak­ing a sola scrip­tura ap­proach, for bet­ter and for worse.

And then you start let­ting the data show you things.

You see things that are sur­pris­ing or odd, and you note that.

You see facts that seem to be in­con­sis­tent with each other, and you look into the data sources and method­ol­ogy un­til you clear up the mys­tery.

You ori­ent to­wards the ran­dom, the un­fa­mil­iar, the things that are to­tally un­fa­mil­iar to your ex­pe­rience. One of the ma­jor ex­ports of Ger­many is valves? When was the last time I even thought about valves? Why valves, what do you use valves in? OK, show me a list of all the differ­ent kinds of ma­chine parts, by per­cent of to­tal ex­ports.

And so, you dig in a lit­tle bit, to this part of the world that you hadn’t looked at be­fore. You cul­ti­vate the abil­ity to spin up a lightweight sort of fan­nish ob­ses­sive cu­ri­os­ity when some­thing seems like it might be a big deal.

And you take ca­sual notes and im­pres­sions (though keep­ing track of all the num­bers and their sources in your notes).

You do a lit­tle bit of ar­ith­metic to com­pare things to fa­mil­iar refer­ence points. How does this source of risk com­pare to the risk of smok­ing or go­ing horse­back rid­ing? How does the effect size of this drug com­pare to the effect size of psy­chother­apy?

You don’t re­ally want to do statis­tics. You might take per­cents, means, stan­dard de­vi­a­tions, maybe a Co­hen’s d here and there, but noth­ing fancy. You’re just try­ing to figure out what’s go­ing on.

It’s of­ten a good idea to rank things by raw scale. What is re­spon­si­ble for the bulk of deaths, the bulk of money moved, etc? What is big? Then pay at­ten­tion more to things, and ask more ques­tions about things, that are big. (Or dis­pro­por­tionately high-im­pact.)

You may find that this pro­cess gives you con­trar­ian be­liefs, but of­ten you won’t, you’ll just have a strongly fact-based as­sess­ment of why you be­lieve the usual thing.

There’s a qual­ity of or­di­nar­i­ness about fact-based be­liefs. It’s not that they’re never sur­pris­ing—they of­ten are. But if you do fact-check­ing fre­quently enough, you be­gin to have a sense of the world over­all that stays in place, even as you dis­cover new facts, in­stead of swing­ing wildly around at ev­ery new stim­u­lus. For ex­am­ple, af­ter do­ing lots and lots of read­ing of the biomed­i­cal liter­a­ture, I have sort of a “sense of the world” of biomed­i­cal sci­ence—what sorts of things I ex­pect to see, and what sorts of things I don’t. My “sense of the world” isn’t that the world it­self is bor­ing—I ac­tu­ally be­lieve in a world rich in dis­cov­er­ies and low-hang­ing fruit—but the sense it­self has sta­bi­lized, feels like “yeah, that’s how things are” rather than “omg what is even go­ing on.”

In ar­eas where I’m less fa­mil­iar, I feel more like “omg what is even go­ing on”, which some­times mo­ti­vates me to go ac­cu­mu­late facts.

Once you’ve ac­cu­mu­lated a bunch of facts, and they’ve “spo­ken to you” with some con­clu­sions or an­swers to your ques­tion, you write them up on a blog, so that other peo­ple can check your rea­son­ing. If your mind gets changed, or you learn more, you write a fol­low-up post. You should, on any topic where you con­tinue to learn over time, feel em­bar­rassed by the naivety of your early posts. This is fine. This is how learn­ing works.

The ad­van­tage of fact posts is that they give you the abil­ity to form in­de­pen­dent opinions based on ev­i­dence. It’s a sort of prac­tice of the skill of see­ing. They likely aren’t the op­ti­mal way to get the most ac­cu­rate be­liefs—listen­ing to the best ex­perts would al­most cer­tainly be bet­ter—but you, per­son­ally, may not know who the best ex­perts are, or may be over­whelmed by the swirl of con­tro­versy. Fact posts give you a rel­a­tively low-effort way of com­ing to in­formed opinions. They make you into the prover­bial ‘ed­u­cated lay­man.’

Be­ing an ‘ed­u­cated lay­man’ makes you much more fer­tile in gen­er­at­ing ideas, for re­search, busi­ness, fic­tion, or any­thing else. Hav­ing facts float­ing around in your head means you’ll nat­u­rally think of prob­lems to solve, ques­tions to ask, op­por­tu­ni­ties to fix things in the world, ap­pli­ca­tions for your tech­ni­cal skills.

Ideally, a group of peo­ple writ­ing fact posts on re­lated top­ics, could learn from each other, and share how they think. I have the strong in­tu­ition that this is valuable. It’s a bit more ac­tive than a “jour­nal club”, and quite a bit more ca­sual than “re­search”. It’s just the ac­tivity of learn­ing and show­ing one’s work in pub­lic.