Epistemic Progress

Epistemic Sta­tus: Cau­tiously op­ti­mistic. Much of this work is in craft­ing and ad­vanc­ing ter­minol­ogy in ways that will hope­fully be in­tu­itive and use­ful. I’m not too at­tached to the speci­fics but hope this could be use­ful for fu­ture work in the area.


Strong epistemics or “good judg­ment” clearly seems valuable, so it’s in­ter­est­ing that it gets rather lit­tle Effec­tive Altru­ist at­ten­tion as a se­ri­ous con­tender for fund­ing and tal­ent. I think this might be a mis­take.

This isn’t to say that epistemics haven’t been dis­cussed. Lead­ers and com­mu­nity mem­bers on LessWrong and the EA Fo­rum have writ­ten ex­ten­sively on epistemic ra­tio­nal­ity, “good judg­ment”, de­ci­sion mak­ing, and so on. Th­ese com­mu­ni­ties seem to have a par­tic­u­lar in­ter­est in “good epistemics.”

But for all the blog posts on the topic, there is less in terms of long term and full-time efforts. We don’t have teams out­lin­ing lists of pos­si­ble large scale epistemic in­ter­ven­tions and es­ti­mat­ing their cost-effec­tive­ness, like an epistemics ver­sion of the Hap­pier Lives In­sti­tute. We don’t have a Global Pri­ori­ties In­sti­tute equiv­a­lent try­ing to for­mal­ize and ad­vance the ideas from The Se­quences. We have very lit­tle work out­line what op­ti­mistic epistemic sce­nar­ios we could hope for 10 to 200 years out from now.

I in­tend to per­son­ally spend a sig­nifi­cant amount of time on these is­sues go­ing for­ward. I have two main goals. One is to bet­ter out­line what I think work in this area could look like and how valuable it might be to pur­sue. The sec­ond is to go about do­ing work in this area in ways that both test the area and hope­fully help lay­out ground­work that makes it eas­ier for more peo­ple to join in.

One pos­si­ble rea­son for a lack of effort in the space is that the cur­rent nam­ing and or­ga­ni­za­tion is a bit of a mess. We have a bun­dle of re­lated terms with­out clear delineations. I imag­ine that if I asked differ­ent peo­ple how they would differ­en­ti­ate “epistemics”, “epistemic ra­tio­nal­ity”, “episte­mol­ogy”, “de­ci­sion mak­ing”, “good judg­ment”, “ra­tio­nal­ity”, “good think­ing”, and the many sub­cat­e­gories of these things, I’d get many con­flict­ing and con­fused an­swers. So some of my goal is to try to high­light some clusters par­tic­u­larly worth pay­ing at­ten­tion to and for­mal­ize what they mean in a way that would be use­ful to make de­ci­sions go­ing for­ward.

I’ll be­gin by in­tro­duc­ing two (hope­fully) self-ev­i­dent ideas. “Epistemic Progress” and “Effec­tive Epistemics.” You can think of “Epistemic Progress” as the “epistemics” sub­set of “Progress Stud­ies”, and “Effec­tive Epistemics” as the epistemic ver­sion of “Effec­tive Altru­ism.” I don’t mean this as an au­thor­i­ta­tive cor­ner­stone, but rather as prag­matic in­tu­itions to get us through the next few posts. Th­ese names are cho­sen mainly be­cause I think they would be the most ob­vi­ous to the au­di­ence I ex­pect to be read­ing this.

Effec­tive Epistemics

Effec­tive Epistemics” is es­sen­tially “what­ever seems to work at mak­ing in­di­vi­d­u­als or groups of peo­ple more cor­rect about things for prag­matic pur­poses.” It’s a bit higher level than Value of in­for­ma­tion. This is not fo­cussed on whether some­thing is the­o­ret­i­cally true or with pre­cise defi­ni­tions of for­mal knowl­edge. It’s rather about which kinds of prac­tices seem to make hu­mans and ma­chines smarter at com­ing to the truth in ways we can ver­ify. If wear­ing pur­ple hats leads to im­prove­ment, that would be part of effec­tive epistemics.

There’s a mul­ti­tude of things that could help or hin­der epistemics. In­tel­li­gence, per­sonal nu­tri­tion, room light­ing, cul­ture, eco­nomic in­cen­tives, math­e­mat­i­cal knowl­edge, ac­cess to ex­per­tise, add-ons to Trello. If “Effec­tive Epistemics” were an aca­demic dis­ci­pline, it wouldn’t at­tempt to en­g­ineer ad­vanced epistemic se­tups, but rather it would sur­vey the space of near and pos­si­ble op­tions to provide or­der­ings. Think “cause pri­ori­ti­za­tion.”

Effec­tive Altru­ism typ­i­cally fo­cuses on max­i­miz­ing the po­ten­tial of large mon­e­tary dona­tions and per­sonal ca­reers. I’d imag­ine Effec­tive Epistemics would fo­cus more on max­i­miz­ing the im­pact of smaller amounts of effort. For ex­am­ple, per­haps it would be iden­ti­fied that if a group of fore­cast­ers all spent 30 hours study­ing in­for­ma­tion the­ory, they could do a 2% bet­ter job in their fu­ture work. My guess is that epistemic in­ter­ven­tion es­ti­ma­tions would be more challeng­ing than hu­man welfare cost-effec­tive­ness calcu­la­tions, so things would prob­a­bly be­gin on a more coarse level. Think longter­mist pri­ori­ti­za­tion (vague and messy), not global welfare pri­ori­ti­za­tion (de­tailed es­ti­mates of lives saved per dol­lar).

Per­haps the most im­por­tant goal for “Effec­tive Epistemics” is to re­ori­ent read­ers to what we care about when we say epistemics. I’m quite para­noid about peo­ple defin­ing epistemics too nar­rowly and ig­nor­ing in­ter­ven­tions that might be wildly suc­cess­ful, but strange.

This para­noia largely comes from the writ­ings of Peter Drucker on hav­ing cor­rect goals, in or­der to ac­tu­ally op­ti­mize for the right things. For ex­am­ple, a school “op­ti­miz­ing ed­u­ca­tion for get­ting peo­ple jobs” might be­gin with High School stu­dents at one point when those are the high­est im­pact. But if things change and they rec­og­nize there are new op­por­tu­ni­ties to ed­u­cate adults, maybe they should jump to pri­ori­tize night school. Per­haps with on­line ed­u­ca­tion they should close down their phys­i­cal build­ing and be­come an on­line-only non­profit fo­cussed on in­ter­na­tional stu­dents with­out great lo­cal schools. It can be very easy to fall into the pat­tern of try­ing to “be a bet­ter con­ven­tional High School than the other con­ven­tional High Schools, on the con­ven­tional mea­sures”, even if what one re­ally cares about upon re­flec­tion is the max­i­miza­tion of value from ed­u­ca­tion.

Epistemic Progress

“Epistemic Progress” points to sub­stan­tial changes in epistemic abil­ities. Progress Stud­ies is an in­ter­est­ing new effort to study the long term progress of hu­man­ity. So far it seems to have a strong em­pha­sis on sci­en­tific and en­g­ineer­ing efforts, which makes a lot of sense as these are very easy to mea­sure over time. There have been a few in­ter­est­ing posts on epistemics but these are a minor­ity. This post on Science in An­cient China seems par­tic­u­larly rele­vant.

His­toric epistemic changes are challeng­ing to define and mea­sure, but they are still pos­si­ble to study. It seems clear in ret­ro­spect that the Re­nais­sance and En­light­en­ment pre­sented sig­nifi­cant gains, and the In­ter­net led to a com­plex mesh of benefits and losses. One should even­tu­ally cre­ate in­dices on “epistemic abil­ities” and track these over time and be­tween com­mu­ni­ties.

One as­pect I’d like to smug­gle into “Epistemic Progress” is a fo­cus on progress go­ing for­ward, or per­haps “epistemic fu­tur­ism”. Epistemic abil­ities might change dra­mat­i­cally in the fu­ture, and it would be in­ter­est­ing to map how that could hap­pen. Epistemic Progress could re­fer to both minor and ma­jor progress, both seem im­por­tant.

Why not fo­cus on [in­sert similar term] in­stead?

I’m not to­tally sure that “epistemics” is the right frame for my fo­cus, as op­posed to the more generic “ra­tio­nal­ity”, or the more spe­cific “in­sti­tu­tional de­ci­sion mak­ing.” As said ear­lier, there are sev­eral over­lap­ping terms float­ing around. There are trade­offs for each.

First, I think it doesn’t re­ally mat­ter. What mat­ters is that we have some com­mon ter­minol­ogy with de­cent enough defi­ni­tions, and use that to pro­duce re­search find­ings. Many of the re­search find­ings should be the same whether one calls the sub­ject “epistemics”, “epistemic ra­tio­nal­ity”, “in­tel­lec­tual de­vel­op­ment”, or so on. If in the fu­ture a more pop­u­lar group comes out with a differ­ent fo­cus, hope­fully, they should make use of the work pro­duced from this line of rea­son­ing. The im­por­tant thing is re­ally that this work gets done, not what we de­cide to call it.

As to why it’s my se­lected choice of the var­i­ous op­tions, I have a few rea­sons. “Epistemics” is an ex­pres­sion with rather pos­i­tive con­no­ta­tions. Hope­fully the choice of “epistemics” vs. “group think­ing” would tilt re­search to fa­vor ac­tors that are well-cal­ibrated in­stead of just be­ing in­tel­li­gent. An in­di­vi­d­ual or group with great de­ci­sion mak­ing or rea­son­ing abil­ities, but sev­eral sub­stan­tial epistemic prob­lems, could do cor­re­spond­ingly se­vere amounts of dam­age. A group with great epistemics could also be de­struc­tive, but a large class of failures (in­tense over­con­fi­dence) may be ex­cluded.

I pre­fer “epistemics” to “de­ci­sion mak­ing” be­cause it gets more to the heart of things. I’ve found when think­ing through the use of Guessti­mate that of­ten by the time you’re mak­ing an ex­plicit de­ci­sion, it’s too late. De­ci­sions are down­stream of gen­eral be­liefs. For ex­am­ple, some­one might make a de­ci­sion to buy a house in or­der to shorten their com­mute, but wouldn’t have ques­tioned whether the wor­ld­view that pro­duced their lifestyle was it­self dra­mat­i­cally sub­op­ti­mal. Per­haps their fun­da­men­tal be­liefs should have been con­tin­u­ously ques­tioned, lead­ing them to forgo their po­si­tion and be­come a Bud­dhist monk.

I’ve been think­ing about us­ing some spe­cific mod­ifier word to differ­en­ti­ate “epistemics” as I re­fer to it from other con­cep­tions. I’m try­ing to go with the col­lo­quial defi­ni­tion that has emerged within the Ra­tion­al­ity and Effec­tive Altru­ist cir­cles, but it should be noted that this defi­ni­tion holds differ­ent con­no­ta­tions to other uses of the term. For this es­say, two new terms feel like enough. I’m go­ing to re­flect on this for fu­ture parts. If you have ideas or prefer­ences, please post them in the com­ments.

Next Steps

Now that we have the key terms, we can start to get into speci­fics. I cur­rently have a rough out­line to for­mally write a few more posts in this se­quence. The broader goal is to help se­cure a foun­da­tion and some mo­ti­va­tion for fur­ther work in this space. If you have thoughts or feed­back, please reach out or post in the com­ments.