Criticism of the average quality of academic research sometimes leaves out two key aspects of the issue. The first author of most academic papers are graduate students and post-docs, who are in the midst of training in the conduct of research. We do not expect people in most professions to start with highly impactful, complex cases. We should take into account the training value of academic research articles as well as the information content. This is explicitly considered when grants are evaluted, but often ignored, even by academics, when reacting to published work that was funded by these grants.
Secondly, published negative results are, individually, typically low impact, despite their potential value for avoiding publication bias and duplication of effort. Positive but low impact results should be seen as being similar in epistemic value to negative results. It would be good if it were more common to accurately describe the impact of one’s work rather than to exaggerate it for publication, and it is necessary to explicitly articulate the true value of the result. However, what I am advocating for is recognizing that, even if there is some exaggeration of the impact in the paper, that even low impact results are still valuable. Perhaps the problem is that we lack adequate understanding or language to address the exponential impact that is possible in scientific research, where modest results can have an impact that is high and nevertheless “below the elbow of the curve.” That may make them suboptimal, but it does not make their existence avoidable and they are not automatically a sign of institutional inadequacy in academia.
My underlying project here is to consider the extent to which academia’s moral legitimacy crisis is due to its own failings, and how much of it is due to knee-jerk criticism by insiders and outsiders alike.
Criticism of the average quality of academic research sometimes leaves out two key aspects of the issue. The first author of most academic papers are graduate students and post-docs, who are in the midst of training in the conduct of research. We do not expect people in most professions to start with highly impactful, complex cases. We should take into account the training value of academic research articles as well as the information content. This is explicitly considered when grants are evaluted, but often ignored, even by academics, when reacting to published work that was funded by these grants.
Secondly, published negative results are, individually, typically low impact, despite their potential value for avoiding publication bias and duplication of effort. Positive but low impact results should be seen as being similar in epistemic value to negative results. It would be good if it were more common to accurately describe the impact of one’s work rather than to exaggerate it for publication, and it is necessary to explicitly articulate the true value of the result. However, what I am advocating for is recognizing that, even if there is some exaggeration of the impact in the paper, that even low impact results are still valuable. Perhaps the problem is that we lack adequate understanding or language to address the exponential impact that is possible in scientific research, where modest results can have an impact that is high and nevertheless “below the elbow of the curve.” That may make them suboptimal, but it does not make their existence avoidable and they are not automatically a sign of institutional inadequacy in academia.
My underlying project here is to consider the extent to which academia’s moral legitimacy crisis is due to its own failings, and how much of it is due to knee-jerk criticism by insiders and outsiders alike.