Musings on LessWrong Peer Review

In the con­text of:

In the spirit of writ­ing down con­ver­sa­tions, this is a rough sum­mary of some re­cent con­ver­sa­tions with Oliver Habryka and Zvi Mow­show­itz about how and why to im­ple­ment peer re­view on Less Wrong.

This is not meant to ar­gue for a point or out­line a spe­cific plan, just to high­light a bunch of the thoughts we’re cur­rently think­ing about. I haven’t put much effort to­wards pol­ish­ing it, but it’s here for peo­ple who want to fol­low along with our thought pro­cess.

Cu­ra­tion, Canon and Com­mon Knowledge

If 90% of the peo­ple around have the idea, when I’m not con­fi­dent that 100% do then I of­ten ex­plain the ba­sic idea for ev­ery­one. This of­ten costs a lot of time.
– Ben Pace on Com­mon Knowledge

Right now we cu­rate about 3 posts a week. This feels about right from the per­spec­tive of read­ers check­ing in on good con­tent reg­u­larly, and au­thors hav­ing a rea­son­able chance of get­ting into cu­rated if their posts are good. But it means cu­ra­tion isn’t that strong a sig­nal of qual­ity or im­por­tance. A 90th per­centile LW post doesn’t nec­es­sar­ily mean “6 months later, we’d ex­pect this idea to still seem im­por­tant.”

Our in­ten­tion was for cu­rated to be a fairly big deal and to only cu­rate things we’re con­fi­dent are “im­por­tant”, but in prac­tice it seems hard.

We’ve con­sid­ered ei­ther rais­ing stan­dards for Cu­rated, or in­tro­duc­ing a new cat­e­gory above it. Early ideas here were re­nam­ing “Cu­rated” to “Com­mon Knowl­edge” with the in­tent to slightly raise stan­dards and im­ply that if you want to stay up to date on “what it’s ex­pected you know if you’re par­ti­ci­pat­ing in LW dis­course”, you should read things in the “Com­mon Knowl­edge” sec­tion.

As cute as this name was, a ma­jor strike against it was that com­mon knowl­edge is a use­ful tech­ni­cal term, and we don’t want to de­grade it’s mean­ing.

We con­sid­ered other terms like “Canon”, with slightly differ­ent con­no­ta­tions: maybe once ev­ery few months we could look back and see which posts (cu­rated or oth­er­wise) seemed like they were likely to en­ter the longterm ra­tio­nal­sphere lex­i­con.

I chat­ted sep­a­rately with Zvi and Oliver about this idea, and found that we had differ­ent in­tu­itions about what “Canon” means.

Canon as History

To Zvi and I, the main as­so­ci­a­tion for canon was “fa­mous works you’re ex­pected to have read, not nec­es­sar­ily be­cause they’re the clear­est or most rigor­ous but be­cause of their his­tor­i­cal con­text.” Maybe some­one has writ­ten a bet­ter ver­sion of Ham­let, or a bet­ter ver­sion of Plato’s Repub­lic, but you may still want to read those if you’re part of sub­cul­tures that value their his­tor­i­cal legacy.

In the ra­tio­nal­sphere, Inad­e­quate Equil­ibria is (sort of) a more for­mal and rigor­ous ver­sion of Med­i­ta­tions on Moloch, but you might still find value in the po­etry, emo­tional oomph and his­tor­i­cal legacy of Med­i­ta­tions, and know­ing about it may help you un­der­stand a lot of con­ver­sa­tions go­ing on since long­time read­ers will be us­ing chunks of it as short­hand.

There­fore, you might want Med­i­ta­tions on Moloch to be rec­og­nized as a part of the Ra­tion­al­ist Canon.

Canon as the Best Reference

Oliver’s take was more like “Canon­i­cal sta­tus is about which writ­ing you want to be the canon­i­cal refer­ence point for a thing”, and you may want this to change over time. (He pointed out the can­on­iza­tion pro­cess of the Bible liter­ally in­volved de­cid­ing which sto­ries seemed im­por­tant, re­vis­ing their or­der, etc)

Part of the rea­son Ben wrote his com­mon knowl­edge post was that the clos­est thing he could find to a canon­i­cal com­mon knowl­edge in­tro­duc­tion was Scott Aaron­son’s es­say, which in­volved a tricky logic puz­zle that I still per­son­ally strug­gle to un­der­stand even af­ter step­ping through it care­fully over sev­eral hours.

There’s value in step­ping through that tricky logic puz­zle, but mean­while, com­mon knowl­edge is a re­ally use­ful con­cept that seemed like it could be ex­plained more in­tu­itively. Ben spent sev­eral weeks writ­ing a post that he hoped stood a chance of be­com­ing the Canon­i­cal Less Wrong Post on Com­mon Knowl­edge.

Peer Re­view as Canon, and Up­ping Our Game

Mean­while, one prob­lem in Real Science™ is that it’s hard to re­move things from canon. Once some­thing’s passed peer re­view, made its way into text­books and en­tered the pub­lic zeit­geist… if you have a repli­ca­tion crisis or a paradigm shift, it may be hard to get peo­ple to re­al­ize the idea is now bo­gus.

This sug­gests two things:

  1. Naively, this sug­gests you need to be re­ally care­ful about what you al­low into Canon in the first place (if us­ing the Canon as Best Refer­ence frame).

  2. You may even want to as­pire higher, to cre­ate a sys­tem where re­mov­ing things from Canon is more in­cen­tivized. This is prob­a­bly harder.

LessWrong 2.0 is aiming to be a plat­form for in­tel­lec­tual progress. Oliver and I are op­ti­mistic about this be­cause we think LessWrong 1.0 con­tributed a lot of gen­uine progress in the fields of ra­tio­nal­ity, effec­tive al­tru­ism, AI safety and x-risk more gen­er­ally. But while promis­ing, the progress we’ve seen so far doesn’t seem as great as we could be do­ing.

In the Canon As His­tory frame, ev­ery­thing in the Se­quences should be part of Canon. In the Canon as Best Refer­ence or Peer Re­view as Canon, frames, there’s… a lot of se­quence posts that might not cut it, for a few rea­sons:

  • The repli­ca­tion crisis hap­pened so some bits of ev­i­dence are not longer as compelling

  • The con­cept hasn’t turned out to be as im­por­tant af­ter sev­eral years of re­fin­ing in­stru­men­tal or epistemic rationality

  • Some of the writ­ing just isn’t that great.

Similarly, in the Slat­es­tar Codex, there’s a lot of posts that in­tro­duce great ideas, but in a highly poli­ti­cized con­text that makes it more fraught to link to them in an un­re­lated dis­cus­sion. It’d be use­ful to have a canon­i­cal refer­ence point that in­tro­duc­ing an idea with­out ril­ing up peo­ple who have strong opinions on fem­i­nism.

And also mean­while, other prob­lems in Real Science™ in­clude the peer re­view be­ing:

  • Thankless for the reviewers

  • In­ter­twined with con­fer­ences and jour­nals that come with weird so­cial games and rent-seek­ing and gate keep­ing problems

  • Qual­ity of re­view varies a lot

  • Some­thing about academia makes peo­ple write in styles that are re­ally hard to un­der­stand, and peo­ple don’t even see this as a prob­lem. (See Chris Olah’s Re­search Debt)

So…

...all of these ideas bump­ing around has me think­ing we shouldn’t just be try­ing to add an­other thresh­old of qual­ity con­trol. As long as we’re do­ing this, let’s try to solve a bunch of long­stand­ing prob­lems in academia, at least in the do­mains that LessWrong has fo­cused on.

I’ve re­cently writ­ten about mak­ing sure LessWrong is friendly to idea gen­er­a­tion. I’ve heard many top con­trib­u­tors talk about feel­ing in­tense pres­sure to make sure their ideas are perfect be­fore try­ing to pub­lish them, and this re­sults in ideas stay­ing locked in­side or­ga­ni­za­tions and in-per­son con­ver­sa­tion.

I think LessWrong still needs more fea­tures that en­able low-pres­sure dis­cus­sion of early stage thoughts.

But, if we’re to be­come a se­ri­ous plat­form for in­tel­lec­tual progress, we also need to in­cen­tivize high cal­iber con­tent that is com­pet­i­tive with main­stream sci­ence and philos­o­phy – some com­bi­na­tion of “as good or bet­ter at rigor and idea gen­er­a­tion” and “much bet­ter at ex­plain­ing things.”

I think a challeng­ing but achiev­able goal might be to be­come some­thing like dis­till.pub, but for the­o­ret­i­cal ra­tio­nal­ity and the as­sorted other top­ics that LessWrongers have ended up in­ter­ested in.

[Note: a differ­ent goal would be to be­come pres­ti­gious in a fash­ion com­pet­i­tive with main­stream sci­ence jour­nals. This seems harder than “be­come a re­li­able in­ter­nal ecosys­tem for the ra­tio­nal­ity and effec­tive al­tru­ism com­mu­ni­ties to de­velop and vet good ideas, with­out re­gard for ex­ter­nal pres­tige.”

I’m not even sure be­com­ing pres­ti­gious would be use­ful, and in­so­far as it is, it seems much bet­ter to try to be­come good and worry about trans­lat­ing that into pres­tige later. Mean­while it’s not a goal I’m per­son­ally in­ter­ested in]


This is the first of a few posts I have planned on this sub­ject. Some up­com­ing con­cepts in­clude:

[Edit: alas, I never wrote these up, al­though we’re still think­ing about it a bunch]

  • What is Peer Re­view for? [The short an­swers are A) high­light­ing the best con­tent and B) forc­ing you to con­tend with feed­back that im­proves your think­ing.]

  • What makes for good crit­i­cism at differ­ent stages of idea development

  • What tools we might build even­tu­ally, and what tools ex­ist now that au­thors may want to use more. (I think a lot of progress can be made just chang­ing some of the as­sump­tions around ex­ist­ing tools on LW and el­se­where)

  • How all of this in­fluences our over­all de­sign choices for the site