Nod posts
Earlier today I was reading How to Read Papers Efficiently: Fast-then-Slow Three pass method. I liked it and upvoted it.
I really like what it’s saying. If you sit down and try to read a paper from start to finish on your first pass, well, that’s just not what you should do. You should skim it first.
After skimming you might determine that it isn’t actually worth your time to delve into further. Sometimes that’s because you’re content with what you learned in the skim. Other times it’s because you realize that it doesn’t actually interest you. And if you do decide to delve into it further, you’ll have an easier time doing so given that you skimmed it first. The high level overview that the skim provides gives you context that makes the subsequent reading easier.
I also really like what it says about doing a third pass after the second. The first pass is to skim, the second is to read, and the third is to understand. After skimming and reading, yeah, you have some sense of what’s going on, but it takes more than that. If you want a deep understanding you’ll have to go back over it again. Probably more than once. I’m surprised that the post didn’t mention this.
And of course, I also really like the zero-ith pass of browsing. Gotta have some sort of filter for what is worth your time to skim.
So yeah, I upvoted this. I like what it’s saying, think it was said reasonably well, and most of all, I think the topic is important.
But then I went for a walk and I found myself rethinking this. Why did I upvote it? Was it really worth it? I started imagining myself texting a friend about it.
Hey man, check out this post. It’s really cool. It talks about how when reading papers you should first skim, then read, then take the time to understand it.
But when I imagine sending that text, I also imagine a response of something like:
Er, I guess so. That seems stuff seems like pretty common knowledge though.
And I’d agree with that. When I read the post, I didn’t actually learn anything new[1]. I just found myself nodding. Nodding pretty hard, but nevertheless, it was just nodding. And a lot of posts that I like do the same thing. They make me nod, but don’t really teach me anything new.
After thinking about this, I decided to take back my upvote. I still do like the post, but I don’t think it passes the threshold of being upvote-worthy.
Similarly, I think that I as well as many others instinctively overvalue these sorts of Nod Posts. It’s not that Nod Posts provide no value. It’s nice to review things that you already know[2]. It’s just that I think we tend to overvalue them.
- ↩︎
I’d expect this to be true for most people, especially smart people. But even for less smart people I think it’s pretty intuitive that you’d skim first, then if something is interesting enough you read it, and then if it’s super fascinating and important you go through it again and again in order to acquire a deep understanding. How People Read Online: New and Old Findings points towards this being true. Maybe this instinct breaks when people are reading things like textbooks or research papers though.
- ↩︎
Reality of course isn’t so cut-and-dried. Sometimes there are hybrids. For example, a Nod Post might be about something that you did already know but haven’t thought about in a while, and it’s useful to bring it back to your attention. I think that’s what this post that I’m writing is probably doing for most people. Other times the post is about something that you understood but it presents it in such a way that deepens your understanding.
Understandable. I’d argue that nod posts should be quite short. But I’ve been disappointed how often what should have been a nod post was new to many folks—hence posting it.
Note that it’s of course meant to be a post of a fairly well known concept. The intent is to highlight “holy crap y’all, please actually get through papers, like, at all”.
Perhaps I need to converse more interactively with those who haven’t been reading papers to figure out why they don’t. Anyone who found the post to be obvious, but who doesn’t read many papers, can you go into detail about why not? I’ve also edited the original post to include a call to debug for those who don’t read many papers.
To be clear, despite de-upvoting, I still do like the post and think it pretty easily passes the threshold of “worth posting” :)
But yeah, that might be true about things that seem like common knowledge actually not being common knowledge. I think I’m a little biased in the direction of assuming it’s common knowledge but I also don’t want to overcorrect, so my best guess is still that the research paper stuff is pretty common knowledge.
Wait, these seem like two different things:
If you’re going to read a paper, how do you go about it.
The decision to read papers at all.
Personally I don’t read any sorts of academic papers because, well, it doesn’t seem like the best use of my time and I don’t find it particularly fun. If I was going to read papers though I’d definitely take the multi-pass approach.
So I’m basically the target audience for the OP—I read a lot, of all kinds of stuff, and almost zero papers. I’m an autodidact with no academic background.
I appreciated the post. I usually need a few reminders that ‘this thing has value’ before I finally get around to exploring it :)
I would say, as the target audience, I’m probably representative when I say that a big part of the reason we don’t read papers is a lack of access, and a lack of discovery tools. I signed up for Elicit a while back, but as above—haven’t gotten around to using it yet :D
Understood!
Hmm, the argument I was imagining myself to be making by making that linkpost (note that it is a linkpost with mild editing to improve reading speed to my & firstuserhere’s eyes) was to attempt to reduce imagined cost of a skim to people who were putting off papers due to thinking they needed to read sequentially. For me, at least, seeing that link a few years ago was revelatory.
Perhaps I do still need to figure out what I want to say about the decision to read papers at all, then. Because it is my view that unless one is reading a lot of papers a month, you’re not going to have a very strong understanding of the manifold of deep learning programs. And they need to be well selected, diverse papers, driven by both curiosity and news, topically relevant, you likely want ai helping you find them. More or less, my view is that the strongest form of cyborgism is to get everyone up to a reading speed where they can get useful alignment-relevant stuff out of skimming research papers not directly in their part of the alignment field.
For some examples of papers I wish more people—especially ones at miri—would at least skim the abstracts to, see my papers posts.
see also this comment
I see. People’s decision/willingness to read papers at all seems like something plausibly worth exploring more. “Talking to users” seems like a good place to start (not that you haven’t already).
Given that a post is new enough to someone for them to upvote it, that probably indicates that it would be new to other people too. It basically becomes a numbers game about how Lesswrong’s infrastructure should be fine-tuned.
I definitely agree with the skimming proposal, the entire system here revolves around people evaluating posts instead of not evaluating them.
I think this undervalues nod posts. Most people don’t hear a good idea once and immediately implement it perfectly. They need reminders, in general and at the right time. Different authors put different spins on posts, a given one may resonate more or less with any particular person. I think the same thing about books that could have been blog posts- the point of the length isn’t to convey more information, it’s to give the same information more real estate in your brain.
Probably most nod posts deserve less karma than the initial post but not always (sometimes the restatement is better, sometimes it’s a new development not a restatement), and less karma doesn’t mean they shouldn’t exist at all.
Yeah, I definitely hear ya. I agree about those benefits you lay out. Despite those benefits, nod posts still strike me as being generally overvalued, but I’m having a hard time coming up with a crux or a way to make progress on the question “are they generally overvalued?”. I think we as well as most people agree on the costs and benefits, but just have different intuitions on how the costs and benefits typically play out.