Content aside, you should generally avoid the first person as well as qualifiers and you should definitely avoid both, e.g. “I think it is interesting.” Where some qualifiers are appropriate, you often phrase them too informally, e.g. “perhaps it is more like,” would read much better as, “It is possible that,” or, “a possible explanation is.” Some first person pronouns are acceptable, but they should really only be used when the only alternative is an awkward or passive sentence.
The beginning paragraph of each subsection should give the reader a clear idea of the ultimate point of that subsection, and you would do well to include a roadmap of everything you plan to cover at the beginning.
I don’t know if this is the feedback you’re searching for or if the writing style is purposeful, just my two cents.
I think how important these criticisms are depends on who the intended audience of the essay is—which Gwern doesn’t really make clear. If it’s basically for SIAI’s internal research use (as you might think, since they paid for it), tone probably hardly matters at all. The same is largely the case if the intended audience is LW users—our preference for accessibly, informally written scholarly essays is revealed by our being LW readers. If it’s meant as a more outward-facing thing, and meant to impress academics who aren’t familiar with SIAI or LW and who judge writings based on their adherence to their own disciplinary norms, then sure. (Incidentally, I do think this would be a worthwhile thing to do, so I’m not disagreeing.) Perhaps Gwern or Luke would care to say who the intended beneficiaries of this article are.
For myself, I prefer scholarly writing that’s as full of first-person statements as the writer cares to make it. I feel like this tends to provide the clearest picture of the writer’s actual thought process, and makes it easier to spot where any errors in thinking actually occurred. I rarely think the accuracy of an article would be improved if the writer went back after writing it and edited out all the first-person statements to make them sound more neutral or universal.
Well, style wasn’t really what I had in mind since it’s already so non-academic in style, but your points are well taken. I’ve fixed some of that.