Reviews as “How should this be improved to meet the bar?”

The LessWrong Review is a pretty big experiment, and I am still very much uncertain what the best form of reviewing is. We were pretty vague on how to review, listing these bullet points on topics a review should consider:

  • How has this post been useful?

  • How does it connect to the broader intellectual landscape?

  • Is this post epistemically sound?

  • How could it be improved?

  • What further work would you like to see on top of the ideas proposed in this post?

I’m interested in seeing more of the last two. I’d like a key piece of the Review to be “authors rewriting posts if they can be improved”, and “people writing followup posts that either attempt a better explanation of whatever the post was about, or do followup work exploring more of the post’s ideas.”

So, for negative reviews, if a post’s central concept seems important but the post itself feels inadequate, I’d be interested in seeing more “how could this post be improved, such that it makes sense to include in a ‘Best Of’, or otherwise enter into Lesswrong’s longterm memory.*”

*a concern I have about the current Review process is it focuses primarily on the best-of book, but not all important works make sense to include in a public facing artifact. Examples of this might include a) dry, technical posts, b) posts that are important but somehow controversial, c) posts that are doing a lot of “schlep” work (such as replications), where you don’t necessarily care that everyone reads them, but you care that they exist and you can refer to them when doing later meta-analyses or whatnot)

I’m not sure about the best solution for this and this is still the subject of some debate on the LessWrong team, but I’d personally prefer if reviewers somewhat distinguished between “whether it makes sense for a post to be in the Best Of Book” and “whether the post is somehow important to the LessWrong intellectual project.”