It would be really nice if studies had a sort of thoroughness check list at the top of the paper next to the abstract clearly stating sample size, sampling process, number of peer reviewers, study methodology(double-blind, panel etc), and any other relevant information to the papers validity. If some sort of crude standardization could even occur within specific fields it would make cross-study comparison much easier. Or what if papers could be published online in a format inviting public criticism and community concerns would be forced to be answered to by authors.
(Sorry, I’ve yet to figure out how to link things and suchlike; can HTML be used here?)
One issue with just allowing anyone to comment on a paper though is a high proportion of misinformed or ignorant people who can hijack the discussion. LW gets round this very well with its judicious gardening, and other sites do this too, so perhaps it’s not as big an issue as I’m making it out to be. Unmoderated comment forums tend to turn into slimepits though.
It would be really nice if studies had a sort of thoroughness check list at the top of the paper next to the abstract clearly stating sample size, sampling process, number of peer reviewers, study methodology(double-blind, panel etc), and any other relevant information to the papers validity. If some sort of crude standardization could even occur within specific fields it would make cross-study comparison much easier. Or what if papers could be published online in a format inviting public criticism and community concerns would be forced to be answered to by authors.
This already happens in some cases. PLoS One, for example, publishes open-access entirely online and invites community criticism:
http://www.plosone.org/static/information.action
(Sorry, I’ve yet to figure out how to link things and suchlike; can HTML be used here?)
One issue with just allowing anyone to comment on a paper though is a high proportion of misinformed or ignorant people who can hijack the discussion. LW gets round this very well with its judicious gardening, and other sites do this too, so perhaps it’s not as big an issue as I’m making it out to be. Unmoderated comment forums tend to turn into slimepits though.