Three Kinds of Research Documents: Clarification, Explanatory, Academic
Epistemic Status: Low. This was a quick idea, but the grouping honesty doesn’t work as well as I’d like. I still think it could be useful to some people though. Ideas appreciated.
Recently I have started writing more and have been trying to be more intentional with what I accomplish. Different documents have different purposes and it seemed useful to help clarify this. Here is a list of three specific different types I think are relevant on LessWrong and similar.
I see clarification posts as generally the first instance of information being written down. Here it is important to get the essential ideas out there and to create consensus around terminology among the most interested readers. In some cases, the only interested reader may be the author, who would use the post just to help cement their ideas for themselves.
Clarification posts may not be immediately useful and require later posts or context for them to make sense. This is typically fine. There’s often not a rush for them to be understood. In many cases, there is a lot of possible information to write down, so the first step is to ensure it’s out there, even if it’s slow, hard to read, or doesn’t much make sense until later.
I think of many of Paul Christiano’s posts as clarification posts. They’re very numerous and novel, but quite confusing to many readers (at least, to myself and several people I’ve talked to). Sometimes the terminology changes from one post to the next. I used to see this is somewhat of a weakness, but now it comes across to me as a pragmatic option. If he were to have tried to make all of this readable to the average LessWrong reader, there’s likely no way he could have written a portion as much.
One important point here is that if something is a clarification post, then the main relevant feedback is on the core content, not the presentation. Giving feedback on the readability can still be useful, but it should be understood and expected that this isn’t the main goal.
Explanatory posts seek to explain content to people. The focus here is on accessibility. Often the main ideas are already documented somewhere, but the author thinks that they could do a better job explaining them to their intended audience.
I would categorize some of the recent posts on Embedded Agency as being explanatory. Some of them have very nice diagrams and are elegantly laid out. I believe much of the content comes from earlier work that was a lot more fragmented and experimental. Zhukeepa’s recent overview of Paul Christiano’s work also is a good example.
Academic documents, as I interpret them, aim to be acceptable to the academic community or considered academic. Some attributes that typically go along with this include:
The academic article structure
Citations, generally of other academic works
Discussion of how work fits in with existing academic literature
A high level of rigor and completeness
An expectation that the main terms and ideas won’t change much
There can definitely be a lot of signaling going on here. Many people see academic seeming articles as substantially more trustworthy and impressive than other works.
That said, I feel like there are some useful attributes to these works besides signaling. For one, it’s a format well suited to interfacing with the academic world. Interfacing with the academic world can be quite valuable, especially in domains with substantial academic work. Also, the format has become popular for some valid reasons around robustness and context.
As an example, MIRI’s official papers fit into this category.
Academic-oriented posts don’t need to be PDFs. I would consider my post on Prediction-Augmented Evaluation Systems to partially be in this category, and several EA Forum posts to partially be in this category (examples here, here, and here.)
There are some documents that do a good job being both “academic” and “explanatory.” I think these should be considered a mix of both.
I think the main take away of this post is that some documents exist for the main purpose of clarification, and should be understood as such. I myself currently have a lot of ideas I want to write down and intend to focus on clarification posts for a while.
The distinction between explanatory and academic documents doesn’t seem as novel nor as elegant to me. I’d be really curious if readers can post in the comments with improvements on this ontology or better examples.