Choice of primary outcome—it’s a good idea to choose a primary outcome measure of relevance. This 6 hour threshold is a little weird (apparently because the study was done in clinical sites not hospitals, so they wanted to get to a sort of hospitalization-equivalent), but they targeted that from the outset, so it’s not that troubling, but maybe not of interest. If the NIH is worried about any ED visit and not just long ones (or doesn’t have a lot of faith in the equivalence), then this measure doesn’t necessarily speak to what the NIH cares about. The NIH barely explains itself there, but TOGETHER may have shot itself in the foot. I’m sure they did the best they could.
NS difference in ITT analysis—I think as-treated (AT) or per-protocol (PP) analyses are generally likeable for studying efficacy as the ITT estimate suffers from contamination, but the NIH’s concern here that it kills the randomization is extremely legitimate. A PP analysis may suffer from self-selection and deserves a little more scrutiny when done not to get a better effect size estimate but push over a statistical significance threshold. The ITT analysis also generates a better estimate for effectiveness in the real world since imperfect adherence happens.
80% threshold for adherence—as you’ve probably noticed, the NIH is having issues with the credibility of the findings. If you are going to do a PP analysis post-hoc, presumably you have a reasonable adherence cutoff, right? Or did you choose that cutoff just to get through the significance filter? Well in this case, the NIH’s concern is a little less fair as the 80% threshold is not atypical for medical research, not that that makes it a great threshold.
Add in a trial to support efficacy on an outcome measure the NIH likes, but it’s small and could suffer from non-response bias with a high amount of attrition. And add in a trial that was stopped for futility. You might update that fluvoxamine has some efficacy and be relieved that it is prescribable, but you might stop short of recommending it as standard of care.
Choice of primary outcome—it’s a good idea to choose a primary outcome measure of relevance. This 6 hour threshold is a little weird (apparently because the study was done in clinical sites not hospitals, so they wanted to get to a sort of hospitalization-equivalent), but they targeted that from the outset, so it’s not that troubling, but maybe not of interest. If the NIH is worried about any ED visit and not just long ones (or doesn’t have a lot of faith in the equivalence), then this measure doesn’t necessarily speak to what the NIH cares about. The NIH barely explains itself there, but TOGETHER may have shot itself in the foot. I’m sure they did the best they could.
NS difference in ITT analysis—I think as-treated (AT) or per-protocol (PP) analyses are generally likeable for studying efficacy as the ITT estimate suffers from contamination, but the NIH’s concern here that it kills the randomization is extremely legitimate. A PP analysis may suffer from self-selection and deserves a little more scrutiny when done not to get a better effect size estimate but push over a statistical significance threshold. The ITT analysis also generates a better estimate for effectiveness in the real world since imperfect adherence happens.
80% threshold for adherence—as you’ve probably noticed, the NIH is having issues with the credibility of the findings. If you are going to do a PP analysis post-hoc, presumably you have a reasonable adherence cutoff, right? Or did you choose that cutoff just to get through the significance filter? Well in this case, the NIH’s concern is a little less fair as the 80% threshold is not atypical for medical research, not that that makes it a great threshold.
Add in a trial to support efficacy on an outcome measure the NIH likes, but it’s small and could suffer from non-response bias with a high amount of attrition. And add in a trial that was stopped for futility. You might update that fluvoxamine has some efficacy and be relieved that it is prescribable, but you might stop short of recommending it as standard of care.