For example: let’s say you want to know the impact of daily jogs on happiness. You randomly instruct 80 people to either jog daily or to simply continue their regular routine. As a per protocol analyst, you drop the many treated people who did not go jogging. You keep the whole control group because it wasn’t as hard for them to follow instructions.
I didn’t realize this was a common practice, that does seem pretty bad!
Do you have a sense of how commonplace this is?
What’s depressing is that there is a known fix for this: intent-to-treat analysis. It looks at effects based on the original assignment, regardless of whether someone complied or not.
In my econometrics classes, we would have been instructed to take an instrumental variables approach, where “assignment to treatment group” is an instrument for “does the treatment”, and then you can use a two stage least squares regression to estimate the effect of treatment on outcome. (My mind is blurry on the details.)
IIUC this sounds similar to intent-to-treat analysis, except allowing you to back out the effect of actually doing the treatment, which is presumably what you care about in most cases.
I also spent a cursed day looking into the literature for NONS. I was going to try and brush this up into a post, but I’m probably not going to do that after all. Here are my scrappy notes if anyone cares to read them.
You’re citing the same main two studies on Enovid that I found (Phase 3 lancet or “Paper 1”, Phase 2 UK trial or “Paper 2″), so in case it’s helpful, here are my notes under “Some concerns you might have” re: the Lancet paper:
The study was funded and conducted by the drug manufacturer (the first 3 authors of the study all work at the manufacturer).
The smaller UK study, which also found that NONS significantly reduced viral load, was conducted by an independent academic researcher.
On the other hand, this study was still funded by the manufacturer, and reports that “the funder of the study conducted the randomisation of the anonymised participant data” (??) which I find pretty weird.
We don’t know about the long term effects — including even relatively short term stuff like whether any patients got rebound COVID after their negative PCR tests.
The study stops following patients after their first negative PCR, so we don’t really have evidence on long term consequences.
Nitrix oxide is supposed to be pretty safe, since it’s produced by your body in response to e.g. eating leafy greens.
But you might worry about mechanism of delivery here — these sprays don’t actually contain NO, they contain agents that combine to produce NO. Might this combination process also create harmful byproducts?
Very very anecdotally, this random science writer on Twitter is worried that combining sodium nitrite in acidic solution could produce nitrosamines, which are carcinogenic. I don’t know any chemistry and can’t quickly check whether this is a valid concern.
The study is supposed to study the effects of nitric oxide. But the treatment spray differs in two ways: it contains NO-creating agents, and an additional gelling agent (HPMC).
This might be a problem as people have suspected HPMC could independently be protective against COVID for a while, and one observational study seems to back this up.
So you might just be picking up on the effects of HPMC, not nitric oxide. Despite being a fairly common food additive, HPMC isn’t well studied as a COVID treatment, so it’s hard to tease these effects out — Here’s a quick summary.
The UK study doesn’t report what exactly they put in the placebo spray, so I can’t verify whether this issue applies to that study.
The in vitro study finds that gaseous NO does not have virucidal effects on SARS-CoV-2, which makes me wonder if in fact HPMC is the real reason why these sprays are effective?
In practice, since there’s only one company that currently sells these sprays, and their product in fact contains HPMC, you might not care about this. But this might tip you in favor of buying a cheaper HPMC spray instead of this product.
.
Note the evidence base on explicitly prophylactic use of NONS is not very good. Here’s the only study I could find (after maybe an hour of searching), and it’s a retrospective epidemiological case study (i.e. not randomly assigned), again by the manufacturers.
They’re running a Phase 3 prophylactic RCT right now, which in theory is supposed to wrap up this month, but who knows when we’ll see the results.