The LessWrong 2022 Review: Review Phase
This year’s LessWrong review nomination phase ended a few days ago, with 339 posts nominated. For comparison, 291 posts were nominated in the 2021 review.
Nomination Phase Results
Here are the current top-20 posts by vote total:
(I’m sensing a bit of a theme...)
More than 60 posts have already been reviewed, but that leaves quite a few posts that have yet to receive any reviews, including many of the most-upvoted ones. If you want to see which posts are most under-reviewed, you can switch your sorting to Magic (Needs Review)[1]. Maybe you have thoughts on Paul’s thoughts on Eliezer’s thoughts?
Inline Reacts!
We’ve got these new nifty inline reacts which you can leave on posts (not just comments!); you may have noticed them. I encourage you to make good use of these when reviewing posts. (Typos should now be a lot less annoying to report, if you’re inclined to do so.)
Prizes? Prizes!
Last year we awarded prizes for good reviews. This year we will also award prizes! We’re aiming for something similar to last year’s, though we haven’t yet worked out the details (size, scope, etc).
Briefly noting what you liked or didn’t like about a post is good. But here are some specific things we’re particularly excited for, from last year
New Information. My favorite reviews are ones that give new information to the reader. This could be a concrete fact, or a rebuttal to a key argument in a post, or a new way the post is relevant that you hadn’t considered. It could be a new frame about how to even think about the post.
Concrete use cases. I think reviews that say “this concretely helped me, here’s how” are also great, especially if they give specifics. I think it’s both helpful and rewarding to individual authors to hear how their post was useful, so they can do more of that. (I think this is also helpful on a broader level, so that the collective LW userbase can see what kinds of effects are realistic)
i.e. rather than saying “this post has helped me a bunch over the years”, say “here are two specific times that it helped me, and how.”
Thoughts on how the post could be improved. Two years later, if a post still seems like good reference material, but is confusing or poorly argued, give advice on how to improve it. Discuss the use-case for the readership you have in mind.
Reflect on the Big Picture. How do various posts fit together into something greater than the sum of their parts? What major conversations have happened on LessWrong and what have you taken from them?
Brevity/clarity. I’m not saying “optimize for shortness at the expense of saying anything substantive”, just, note that all-else-being-equal, taking less space to convey the key information is helpful.
Final Voting
The review phase ends on January 14th, which is when final voting starts.
- ^
Or click that link!
First round of review prizes for this year:
$200:
@Buck for his self-review on the Causal Scrubbing post
@ambigram for their review of Meadow Theory
@LoganStrohl for their self-review of the Naturalism sequence
@porby for their self-review of “Why I think strong general AI is coming soon”
$100:
@Lee Sharkey for their self-review of the sparse auto-encoder post
@Writer for their review of “Inner and outer alignment decompose one hard problem into two extremely hard problems”
@Neel Nanda for their review of “Some Lessons Learned from Studying Indirect Object Identification in GPT-2 small”
@janus for their self-review of the Simulators post
$50:
@Davidmanheim for their review of “It’s Probably not Lithium”
@Vika for their self-review of “DeepMind alignment team opinions on AGI ruin arguments”
@TurnTrout for their review of “Simualators”
@Jan_Kulveit for their review of “Human values & biases are inaccessible to the genome”
@Alex_Altair for their review of the Naturalism sequence
@Neel Nanda for their self-review of “A Longlist of Theories of Impact for Interpretability”
@Screwtape for their review of “Sazen”
@eukaryote for their self-review of “Fiber arts, mysterious dodecahedrons, and waiting on “Eureka!””
@L Rudolf L for their self-review of “Review: Amusing Ourselves to Death”
@Vika for their self-review of “Refining the Sharp Left Turn threat model, part 1: claims and mechanisms”
@[DEACTIVATED] Duncan Sabien for their review of “Here’s the exit.”
What are you looking for in reviews? What makes for good ones?
I just edited the post to include a recap of some things that make a good review,
For clarity, are the review & post results going to be separated into AI and non-AI stuff again, like they IIRC were in some previous year? I’d like to review some non-AI posts, but wouldn’t bother doing so if all the top spots are going to be won by AI posts by default.
I’m not sure this question is that meaningful – we carve up the results in different ways for different purposes. In previous years we’ve put the top 50 things into books. We may not do exactly that again, but I think taking stock of the top 50 posts is still a worthwhile thing to do. Epistemic Legibility clocks in at #8 followed by non-AI posts. I think the non-AI content from the year was still doing something meaningful I want people to think about and process.
(Or, to put another way: if it turned out that not dividing up the categories was making people feel unmotivated to do non-AI posts, I’d probably go ahead and break them up into categories, because it does seem important to me)
I meant for stuff like prizes etc.
Overall my attitude is like, if the top 7 posts are all AI posts, that’s not because they’re necessarily better than the best non-AI posts, but rather because AI has been The Topic since 2022, plus the readership has dramatically shifted towards AI content. At which point we might as well declare LW to be a full-time AI site and consider all the rest to be mere hobbyist content =(. Such a ranking outcome would disincentivize authors from writing about the latter. Better to split the ranking into two top-25s or something.
Also, if the rankings are not split up, then if one only visits LW for AI or non-AI content, that gives an annoying strategic incentive to review-downvote all the other content. That doesn’t occur if the rankings are separate.
Ah, gotcha. Yeah this does update me that we should split out the prizes in some way along the lines you’re suggesting here. (I’m not involved with final decisions on the Review this year since I’m focused on some other projects, but this does seem like an issue)