Academics know, or at least ought to know, that most new publications are either wrong or completely uninteresting. This is the logical side-effect from the publish-or-perish policy in academia and the increase of PhD students worldwide. Estimated 1.346 million papers per year are published in journals alone[1]. If humanity produced interesting papers at that rate scientific progress would go a lot quicker!
So if it’s true that most publications are uninteresting and if it’s true that most academics have to publish at a high rate in order to protect their career and send the right signals we don’t want to punish and humiliate academics for publishing stupid ideas or badly executed experiments. And when you publish a paper that demonstrates the other party did a terrible job it does exactly that. The signal to noise ratio in academic journals wouldn’t increase by much but suddenly academics can simply reach their paper quota by picking the ideas of other academics apart. You’d get an even more poisonous environment as a result!
In our current academic environment (or at least my part of it) most papers without a large number of citations are ignored. A paper without any citations is generally considered such a bad source that it’s only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it. So in practice I don’t think it’s a big deal that mistakes aren’t corrected and that academics typically aren’t expected to publicly admit that they were wrong. It’s just not necessary.
A paper without any citations is generally considered such a bad source that it’s only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it. So in practice I don’t think it’s a big deal that mistakes aren’t corrected and that academics typically aren’t expected to publicly admit that they were wrong. It’s just not necessary
Suppose the paper supposedly proves something that lots of people wish was true. Surely it is likely to get an immense number of citations.
For example,the paper supposedly proves that America always had strict gun control, or that the world is doomed unless government transfers trillions of dollars from group A to group B, by restricting the usage of evil substance X, where group A tends to have rather few academics, and group B tends to have rather a lot of academics.
So if it’s true that most publications are uninteresting and if it’s true that most academics have to publish at a high rate in order to protect their career and send the right signals we don’t want to punish and humiliate academics for publishing stupid ideas or badly executed experiments. And when you publish a paper that demonstrates the other party did a terrible job it does exactly that. The signal to noise ratio in academic journals wouldn’t increase by much but suddenly academics can simply reach their paper quota by picking the ideas of other academics apart
Surely it’s better to have academics picking apart crap than producing crap.
Not necessarily. Ignoring crap may be a better strategy than picking it apart.
Cooperation is also easier when different groups in the same research area don’t try too hard to invalidate each other’s claims. If the problem in question is interesting you’re much better off writing your own paper on it with your own claims and results. You can dismiss the other paper with a single paragraph: “Contrary to the findings of I.C. Wiener in [2] we observe that...” and leave it at that.
The system is entirely broken but I don’t see an easy way to make it better.
A paper without any citations is generally considered such a bad source that it’s only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it.
If this were true how would anyone ever get the first citation?
(Incidentally in my own field, there are a lot of papers that don’t get cited. It isn’t because the papers are wrong (although some very small fraction of them have that problem) but that they just aren’t interesting. But math is very different from most other fields.)
If this were true how would anyone ever get the first citation?
Some papers (those written by high status authors) are ones that everyone knows will get citations soon after they are published, and so they feel safe in citing them since others are soon to do so. Self-fulfilling prophecy.
If this were true how would anyone ever get the first citation?
Because the policy wasn’t applied until after a cutoff date, so the recursion bottoms out at an author from before the cutoff. Obviously. Edit: Non-obviously. Edit2: HOW AM I SUPPOSED TO END THIS COMMENT FOR YOU MEATBAGS NOT TO VOTE ME DOWN???
I think your comment is getting voted down because it doesn’t actually answer the issue in question. It does allow there to be a set of citable papers, but it doesn’t deal with the actual question which is how any given paper would ever get its first citation.
Yes, it does, because paper B, from after the cutoff, cites a cite-less paper A, from before the cutoff. Then a paper C can cite B (or A), as B cites a previous paper, and A is from a time for which the standard today is not applied. (Perhaps I wasn’t clear that the cutoff also applies to citable papers—papers from before the cutoff don’t themselves need citations in them to be citable.)
Edit: Also, papers from before the cutoff cited other prior papers.
It’s not citing but being cited, I think. So if A and B are both before the cutoff, and A cites B, then C from after the cutoff can cite B (but not necessarily A).
If this were true how would anyone ever get the first citation?
Zed didn’t say you should never cite a previously uncited paper, only that you shouldn’t invest time and effort into work that depends on the assumption that its conclusions are sound. There are many possible reasons why you might nevertheless want to cite it, and perhaps even give it some lip service.
I’m confused. I parsed this as “papers which contain no citations are considered bad sources,” but it seems that everyone else is parsing it as “papers which have not been cited are considered bad sources.” Am I making a mistake here? The latter doesn’t make much sense to me, but Zed hasn’t stepped in to correct that interpretation.
Look at the context of the first two paragraphs and the comment that Zed was replying to. The discussion was about how many papers never get cited at all. In that context, he seems to be talking about people not citing papers unless they have already been cited.
It’s not clear to me that he was talking about studies being ignored because they’re not interesting enough to cite, rather than studies being ignored because they’re not trustworthy enough to cite.
In the case of only citing papers that contain numerous citations, this is helpful if the papers contain many redundant citations, demonstrating that the factual claims have been replicated, but if a paper relies on many uncertain findings, then its own uncertainty will be multiplied. The conclusion is at most as strong as its weakest link.
If this were true how would anyone ever get the first citation?
I think that you can cite it if you really think that it’s good (and perhaps this is what Zed meant by “if you must”), but you’d better be citing something more widely accepted too. Then if lots of people think that it’s really good, it will join the canon of widely cited papers.
Also, people outside of the publish-or-perish framework (respected elders, hobbyists outside of the research track, grad students in some fields) can get the ball rolling.
Academics know, or at least ought to know, that most new publications are either wrong or completely uninteresting. This is the logical side-effect from the publish-or-perish policy in academia and the increase of PhD students worldwide. Estimated 1.346 million papers per year are published in journals alone[1]. If humanity produced interesting papers at that rate scientific progress would go a lot quicker!
So if it’s true that most publications are uninteresting and if it’s true that most academics have to publish at a high rate in order to protect their career and send the right signals we don’t want to punish and humiliate academics for publishing stupid ideas or badly executed experiments. And when you publish a paper that demonstrates the other party did a terrible job it does exactly that. The signal to noise ratio in academic journals wouldn’t increase by much but suddenly academics can simply reach their paper quota by picking the ideas of other academics apart. You’d get an even more poisonous environment as a result!
In our current academic environment (or at least my part of it) most papers without a large number of citations are ignored. A paper without any citations is generally considered such a bad source that it’s only one step up from wikipedia. You can cite it, if you must, but you better not base your research on it. So in practice I don’t think it’s a big deal that mistakes aren’t corrected and that academics typically aren’t expected to publicly admit that they were wrong. It’s just not necessary.
Suppose the paper supposedly proves something that lots of people wish was true. Surely it is likely to get an immense number of citations.
For example,the paper supposedly proves that America always had strict gun control, or that the world is doomed unless government transfers trillions of dollars from group A to group B, by restricting the usage of evil substance X, where group A tends to have rather few academics, and group B tends to have rather a lot of academics.
Surely it’s better to have academics picking apart crap than producing crap.
Not necessarily. Ignoring crap may be a better strategy than picking it apart.
Cooperation is also easier when different groups in the same research area don’t try too hard to invalidate each other’s claims. If the problem in question is interesting you’re much better off writing your own paper on it with your own claims and results. You can dismiss the other paper with a single paragraph: “Contrary to the findings of I.C. Wiener in [2] we observe that...” and leave it at that.
The system is entirely broken but I don’t see an easy way to make it better.
If this were true how would anyone ever get the first citation?
(Incidentally in my own field, there are a lot of papers that don’t get cited. It isn’t because the papers are wrong (although some very small fraction of them have that problem) but that they just aren’t interesting. But math is very different from most other fields.)
Some papers (those written by high status authors) are ones that everyone knows will get citations soon after they are published, and so they feel safe in citing them since others are soon to do so. Self-fulfilling prophecy.
Because the policy wasn’t applied until after a cutoff date, so the recursion bottoms out at an author from before the cutoff. Obviously. Edit: Non-obviously. Edit2: HOW AM I SUPPOSED TO END THIS COMMENT FOR YOU MEATBAGS NOT TO VOTE ME DOWN???
I don’t know, but I’m pretty sure that’s not it.
I think your comment is getting voted down because it doesn’t actually answer the issue in question. It does allow there to be a set of citable papers, but it doesn’t deal with the actual question which is how any given paper would ever get its first citation.
Yes, it does, because paper B, from after the cutoff, cites a cite-less paper A, from before the cutoff. Then a paper C can cite B (or A), as B cites a previous paper, and A is from a time for which the standard today is not applied. (Perhaps I wasn’t clear that the cutoff also applies to citable papers—papers from before the cutoff don’t themselves need citations in them to be citable.)
Edit: Also, papers from before the cutoff cited other prior papers.
It’s not citing but being cited, I think. So if A and B are both before the cutoff, and A cites B, then C from after the cutoff can cite B (but not necessarily A).
Personally I thought it was a good comment even before the edit.
Zed didn’t say you should never cite a previously uncited paper, only that you shouldn’t invest time and effort into work that depends on the assumption that its conclusions are sound. There are many possible reasons why you might nevertheless want to cite it, and perhaps even give it some lip service.
Especially if it’s your own.
Self-citations are usually counted separately (both for formal purposes and in informal assessments of this sort).
I’m confused. I parsed this as “papers which contain no citations are considered bad sources,” but it seems that everyone else is parsing it as “papers which have not been cited are considered bad sources.” Am I making a mistake here? The latter doesn’t make much sense to me, but Zed hasn’t stepped in to correct that interpretation.
Look at the context of the first two paragraphs and the comment that Zed was replying to. The discussion was about how many papers never get cited at all. In that context, he seems to be talking about people not citing papers unless they have already been cited.
It’s not clear to me that he was talking about studies being ignored because they’re not interesting enough to cite, rather than studies being ignored because they’re not trustworthy enough to cite.
In any case, I think both are dubious safety mechanisms. John Ioannidis found that even most the most commonly cited studies in medical research are highly likely to be false. If researchers are basing their trust in studies on the rate at which they’re cited, they’re likely to be subject to information cascades, double counting the information that led other researchers to cite the same study.
In the case of only citing papers that contain numerous citations, this is helpful if the papers contain many redundant citations, demonstrating that the factual claims have been replicated, but if a paper relies on many uncertain findings, then its own uncertainty will be multiplied. The conclusion is at most as strong as its weakest link.
I think that you can cite it if you really think that it’s good (and perhaps this is what Zed meant by “if you must”), but you’d better be citing something more widely accepted too. Then if lots of people think that it’s really good, it will join the canon of widely cited papers.
Also, people outside of the publish-or-perish framework (respected elders, hobbyists outside of the research track, grad students in some fields) can get the ball rolling.