In my former area (hep-th, gr-qc, quant-ph) a lot of research is open-loop, speculative calculations, with no way to check even the basic assumptions. The first-rate research is reasonably easy to identify, but the rest is something like “we take this totally made-up model from some other paper, make more questionable simplifying or complicating assumptions, calculate some amplitudes, diagrams, propagators, BPS degeneracy, dual this or that”. Unless you work on that particular topic, it is almost impossible to evaluate anything other than the math part, if that. Then it gets published, since the general area of research is fashionable, and there is nothing obviously wrong with the calculations, unrealistic though may be. So the “fraud” is basically writing something that could be confused with snarxiv, rather than falsifying anything outright. So the main issue is of noise, rather than of deliberate fraud.
Re payment to reviewers: I cannot see this working in any remotely desirable direction. Instead there would be people making money by writing positive reviews, like the shills on YELP and such.
I agree there is a problem, but the only suggestion I have is that arxiv should allow (semi-anonymous) comments by the experts in the area. Something Reddit-like, maybe.
A comment system is a whole ’nother ball of wax. Who is qualified to comment? Who moderates the comments? What kind of moderation is used? Experience suggests that the comments section—even if thoroughly moderated to get rid of spam and trolls—will become a place where people insert opinions that are mostly either completely unrelated to the article, or result from not reading the article, or just downright wrong. A possibility would be non-anonymous comments based on some sort of reputation system like the various stackoverflow-related sites.
In my former area (hep-th, gr-qc, quant-ph) a lot of research is open-loop, speculative calculations, with no way to check even the basic assumptions. The first-rate research is reasonably easy to identify, but the rest is something like “we take this totally made-up model from some other paper, make more questionable simplifying or complicating assumptions, calculate some amplitudes, diagrams, propagators, BPS degeneracy, dual this or that”. Unless you work on that particular topic, it is almost impossible to evaluate anything other than the math part, if that. Then it gets published, since the general area of research is fashionable, and there is nothing obviously wrong with the calculations, unrealistic though may be. So the “fraud” is basically writing something that could be confused with snarxiv, rather than falsifying anything outright. So the main issue is of noise, rather than of deliberate fraud.
Re payment to reviewers: I cannot see this working in any remotely desirable direction. Instead there would be people making money by writing positive reviews, like the shills on YELP and such.
I agree there is a problem, but the only suggestion I have is that arxiv should allow (semi-anonymous) comments by the experts in the area. Something Reddit-like, maybe.
A comment system is a whole ’nother ball of wax. Who is qualified to comment? Who moderates the comments? What kind of moderation is used? Experience suggests that the comments section—even if thoroughly moderated to get rid of spam and trolls—will become a place where people insert opinions that are mostly either completely unrelated to the article, or result from not reading the article, or just downright wrong. A possibility would be non-anonymous comments based on some sort of reputation system like the various stackoverflow-related sites.
yes, I mean the latter, reputation-based, but potentially anonymized, so that only serious replies are allowed.