Hypothetically, what would the right response be if you noticed that one of the main vaccine trials has really terrible blinding (e.g. participants are talking about how to tell whether you get the placebo in the waiting room)?
It seems like it would really mess up the data, probably resulting in the people who got the the vaccine taking extra risk and leading the study to understate the effectiveness. Ideally, “tell the researchers” would be the obvious right answer, but are there perverse incentives at play that make the best response something else?
If I didn’t have people thanking me every week for doing these, it would be difficult to keep going.
Tell the researchers seems right. But I’d note that if blinding is bad, that seems like it would make the control group take relatively less risk, so the results should still be valid?
I presume the blinding is imperfect because some of the vaccines cause mild reactions that the placebo wouldn’t. I doubt it’s a big problem. The people doing the trial are selected for being more conscientious than the average person. (For one of the two trials, the rate of Covid seropositivity was only ~1% for people starting the trial, which is lower than the general US population). They will not want to risk their household members getting Covid, and they will have been warned that that the vaccines are unlikely to work perfectly.
Researchers should 1) survey participants regarding their possible exposure risks, and 2) ask them whether they think they got the placebo, and with what degree of confidence. Adjusting for these should reduce the problem.
Hypothetically, what would the right response be if you noticed that one of the main vaccine trials has really terrible blinding (e.g. participants are talking about how to tell whether you get the placebo in the waiting room)?
It seems like it would really mess up the data, probably resulting in the people who got the the vaccine taking extra risk and leading the study to understate the effectiveness. Ideally, “tell the researchers” would be the obvious right answer, but are there perverse incentives at play that make the best response something else?
Thanks Zvi. The effort is definitely appreciated.
Tell the researchers seems right. But I’d note that if blinding is bad, that seems like it would make the control group take relatively less risk, so the results should still be valid?
I presume the blinding is imperfect because some of the vaccines cause mild reactions that the placebo wouldn’t. I doubt it’s a big problem. The people doing the trial are selected for being more conscientious than the average person. (For one of the two trials, the rate of Covid seropositivity was only ~1% for people starting the trial, which is lower than the general US population). They will not want to risk their household members getting Covid, and they will have been warned that that the vaccines are unlikely to work perfectly.
Researchers should 1) survey participants regarding their possible exposure risks, and 2) ask them whether they think they got the placebo, and with what degree of confidence. Adjusting for these should reduce the problem.