Yes, but then you have to send the researchers to India. (Unless you also recruit Indian psychologists who already live there to do your replications.)
Gap year students! They can dig some irrigation ditches while they’re there.
Critically, they don’t need to devise their own experiments; they’re effectively doing the leg-work for more senior researchers back in the UK/US, and also making use of the language & cultural skills they’ve learnt for their gap year/volunteering. Also, the data they gather can be used both to judge the hypothesis the test was originally investigating, and reveal differences between cultures and nations.
Ooh, this could be a scholarship thing. “Study Abroad And Do Replication Studies Fund”. Give ’em a grand apiece, no essay required, I bet it would work.
I’d be worried about trusting the students. It’s like giving them a test and your answer key, and telling them ‘hey, we did our best in getting the right answers, but please work through all the problems again and see whether we made any mistakes’. This sort of thing only works if you don’t get too much garbage in your replications.
The students might be honest enough to actually do all the work professionally, but I’m not sure I’d trust American students (a summer/semester isn’t that long, and if they’re in India, there are things to do there that could fill a lifetime; the temptation to just fudge up some data and go do all those awesome things would be tremendous), much less Indian ones.
This sort of thing only works if you don’t get too much garbage in your replications.
You have way too much trust in the professors. Just a few students naive enough to do what they’re supposed to would be an improvement on the status quo.
“We targeted 13 gene-disease associations, each already assessed by meta-analyses, including at least 15 non-Chinese studies. We searched the Chinese Journal Full-Text Database for additional Chinese studies on the same topics. We identified 161 Chinese studies on 12 of these gene-disease associations; only 20 were PubMed-indexed (seven English full-text). Many studies (14–35 per topic) were available for six topics, covering diseases common in China. With one exception, the first Chinese study appeared with a time lag (2–21 y) after the first non-Chinese study on the topic. Chinese studies showed significantly more prominent genetic effects than non-Chinese studies, and 48% were statistically significant per se, despite their smaller sample size (median sample size 146 versus 268, p < 0.001). The largest genetic effects were often seen in PubMed-indexed Chinese studies (65% statistically significant per se). Non-Chinese studies of Asian-descent populations (27% significant per se) also tended to show somewhat more prominent genetic effects than studies of non-Asian descent (17% significant per se).”
The huge amount of data that could be gathered should allow for checking; data that is both different from what westerners would expect, and consistent over several independent students, is likely to be accurate. Or at least, not inaccurate because of lazy students.
Yes, but then you have to send the researchers to India. (Unless you also recruit Indian psychologists who already live there to do your replications.)
Gap year students! They can dig some irrigation ditches while they’re there.
Critically, they don’t need to devise their own experiments; they’re effectively doing the leg-work for more senior researchers back in the UK/US, and also making use of the language & cultural skills they’ve learnt for their gap year/volunteering. Also, the data they gather can be used both to judge the hypothesis the test was originally investigating, and reveal differences between cultures and nations.
Ooh, this could be a scholarship thing. “Study Abroad And Do Replication Studies Fund”. Give ’em a grand apiece, no essay required, I bet it would work.
I’d be worried about trusting the students. It’s like giving them a test and your answer key, and telling them ‘hey, we did our best in getting the right answers, but please work through all the problems again and see whether we made any mistakes’. This sort of thing only works if you don’t get too much garbage in your replications.
The students might be honest enough to actually do all the work professionally, but I’m not sure I’d trust American students (a summer/semester isn’t that long, and if they’re in India, there are things to do there that could fill a lifetime; the temptation to just fudge up some data and go do all those awesome things would be tremendous), much less Indian ones.
You have way too much trust in the professors. Just a few students naive enough to do what they’re supposed to would be an improvement on the status quo.
The problem is, we already have replications being done by Indian and Chinese scientists and… they’re not very good. Here’s one: “Local Literature Bias in Genetic Epidemiology: An Empirical Evaluation of the Chinese Literature”, 2005:
The huge amount of data that could be gathered should allow for checking; data that is both different from what westerners would expect, and consistent over several independent students, is likely to be accurate. Or at least, not inaccurate because of lazy students.