You suggest “professional subjects” would try to decide whether they’re in the placebo or treatment arm. But this seems difficult (people who are badly-off enough to consider “professional patient” a career choice probably are going to have a hard time testing for chemical compositions, and as you mention placebos can cause placebo side effects), unnecessary (researchers can never say “You didn’t give us the effect we expected, so we think you’re cheating and we’re not going to pay you for this study and we’re blacklisting you for future studies” without opening a can of worms), and contradictory to the observed results (if fake patients did this successfully, it would mean a greater difference between placebo and experimental, since placebo would report no change and experimental would report strong positive effects, but the conundrum to be explained is that antidepressant studies seem to be showing less efficacy over time.)
I think it’s much more plausible that if these “professional subjects” existed, they would just say “Yes, the drug worked great” on the theory that this is what the researchers want to hear (they may not understand the idea of placebo-control, and even if they do, they know that it means they can totally get away with being “cured” by a fake pill). This would present as placebos appearing more effective than they did in previous, more honest trials—which seems to be happening—and a decrease in positive findings.
I would be surprised if this were large enough to make a big difference, but it’s certainly worth considering. I wonder if there are any meta-analyses comparing studies recruiting by public advertising, versus studies recruiting by a doctor or institution referring their own patient population.
(researchers can never say “You didn’t give us the effect we expected, so we think you’re cheating and we’re not going to pay you for this study and we’re blacklisting you for future studies” without opening a can of worms)
I linked the previous post on my Facebook and got this comment from Paul Riddell:
“This reminds me of tales related by friends living in Austin, which has both the University of Texas and a big pharmaceutical testing facility. Although researchers would swear up and down that they’d never withhold payment for testing if a test subject showed, say, severe allergic reations, it happened all of the time. To that end, subjects learned very early on, and they’d pass it on to friends in need of cash, that they should never report adverse symptoms. And now we see the legacy of this, with lots of very disturbing side effects only now being recognized from the line of antidepressants tested there.”
The research itself is full of bad incentives. Ben Goldacre has frequently suggested a database of all clinical trials, including failed ones. He has a raft of newspaper columns on the hows and whys of bad science in medicine. Professional patients are the very least of it.
Back when there were no good treatments for AIDS and it was regarded as pretty much a death sentence, patients would in fact conspire to see who was getting the drug or placebo, and drop out if they had the placebo and try to sign up for a study with the drug. This of course misses the point of an RCT, but the point is that desperate people aren’t in it for the science.
You suggest “professional subjects” would try to decide whether they’re in the placebo or treatment arm. But this seems difficult (people who are badly-off enough to consider “professional patient” a career choice probably are going to have a hard time testing for chemical compositions, and as you mention placebos can cause placebo side effects), unnecessary (researchers can never say “You didn’t give us the effect we expected, so we think you’re cheating and we’re not going to pay you for this study and we’re blacklisting you for future studies” without opening a can of worms), and contradictory to the observed results (if fake patients did this successfully, it would mean a greater difference between placebo and experimental, since placebo would report no change and experimental would report strong positive effects, but the conundrum to be explained is that antidepressant studies seem to be showing less efficacy over time.)
I think it’s much more plausible that if these “professional subjects” existed, they would just say “Yes, the drug worked great” on the theory that this is what the researchers want to hear (they may not understand the idea of placebo-control, and even if they do, they know that it means they can totally get away with being “cured” by a fake pill). This would present as placebos appearing more effective than they did in previous, more honest trials—which seems to be happening—and a decrease in positive findings.
I would be surprised if this were large enough to make a big difference, but it’s certainly worth considering. I wonder if there are any meta-analyses comparing studies recruiting by public advertising, versus studies recruiting by a doctor or institution referring their own patient population.
I linked the previous post on my Facebook and got this comment from Paul Riddell:
“This reminds me of tales related by friends living in Austin, which has both the University of Texas and a big pharmaceutical testing facility. Although researchers would swear up and down that they’d never withhold payment for testing if a test subject showed, say, severe allergic reations, it happened all of the time. To that end, subjects learned very early on, and they’d pass it on to friends in need of cash, that they should never report adverse symptoms. And now we see the legacy of this, with lots of very disturbing side effects only now being recognized from the line of antidepressants tested there.”
That is seriously f’ed up.
Likewise, the fact that we’re better at checking for serial credit card applicants than serial test patients.
The research itself is full of bad incentives. Ben Goldacre has frequently suggested a database of all clinical trials, including failed ones. He has a raft of newspaper columns on the hows and whys of bad science in medicine. Professional patients are the very least of it.
Back when there were no good treatments for AIDS and it was regarded as pretty much a death sentence, patients would in fact conspire to see who was getting the drug or placebo, and drop out if they had the placebo and try to sign up for a study with the drug. This of course misses the point of an RCT, but the point is that desperate people aren’t in it for the science.