Wikipedia addresses this… I was just reading the wiki on the Paleo diet and saw a bunch of stuff about repeatability and study relevance like:
Loren Cordain, a proponent of a low-carbohydrate Paleolithic diet, responded to the U.S. News ranking, stating that their “conclusions are erroneous and misleading” and pointing out that “five studies, four since 2007, have experimentally tested contemporary versions of ancestral human diets and have found them to be superior to Mediterranean diets, diabetic diets and typical western diets in regards to weight loss, cardiovascular disease risk factors and risk factors for type 2 diabetes.”[27] The editors of U.S. News replied that their ranking included a review of all five studies which found that all of them were small and/or of short duration.
I realize Wikipedia isn’t credible for citing or anything but I feel heartened because:
I bet they often link to a credible meta-analysis, making it easier to find them (I’ve been told by Gwern that one way of coping with this is to read a meta-analysis because it gives you a number of advantages over reading individual pieces of research).
It serves as a method for finding out about some of the flaws you need to look for when reading studies on the topic.
It often lists a collection of relevant research, which can save time.
It might be a good starting point for creating your own thorough reviews of studies because a lot of things will already have been hashed out, so it’s just a matter of verifying that what’s there is correct, which should save time if you build on it.
Hm...
Wikipedia is not a perfect solution but I think this will help me cope.
.oO I wonder if there are features that could be added to Wikipedia that would encourage the entries to transform into credible meta-analyses...
A very good Wikipedia article will be equivalent to a review article, but such an article isn’t a meta-analysis: it doesn’t include only studies which can be boiled down to a few summary statistics like d. There’s also little way of being sure that the article is comprehensive and unbiased—one reason meta-analyses usually make a point of how they did a big search on Pubmed and looked through hundreds of results etc.
I don’t know what features could be added to deal with either problem. Any meta-analyses tucked into WP articles would be rightly considered Original Research.
suddenly thinks of a coping strategy
Wikipedia addresses this… I was just reading the wiki on the Paleo diet and saw a bunch of stuff about repeatability and study relevance like:
I realize Wikipedia isn’t credible for citing or anything but I feel heartened because:
I bet they often link to a credible meta-analysis, making it easier to find them (I’ve been told by Gwern that one way of coping with this is to read a meta-analysis because it gives you a number of advantages over reading individual pieces of research).
It serves as a method for finding out about some of the flaws you need to look for when reading studies on the topic.
It often lists a collection of relevant research, which can save time.
It might be a good starting point for creating your own thorough reviews of studies because a lot of things will already have been hashed out, so it’s just a matter of verifying that what’s there is correct, which should save time if you build on it.
Hm...
Wikipedia is not a perfect solution but I think this will help me cope.
.oO I wonder if there are features that could be added to Wikipedia that would encourage the entries to transform into credible meta-analyses...
A very good Wikipedia article will be equivalent to a review article, but such an article isn’t a meta-analysis: it doesn’t include only studies which can be boiled down to a few summary statistics like d. There’s also little way of being sure that the article is comprehensive and unbiased—one reason meta-analyses usually make a point of how they did a big search on Pubmed and looked through hundreds of results etc.
I don’t know what features could be added to deal with either problem. Any meta-analyses tucked into WP articles would be rightly considered Original Research.