Short version: the underlying paper is methodologically honest and reasonably well-run, but its evidence for acupuncture having a specific (beyond-placebo) effect is weak. The National Geographic article substantially oversells it — it leads with a fragile secondary finding and quietly drops the fact that the trial’s primary outcome was null.
The pre-registered primary outcome was vulvar pain (Average Pain Intensity) at end of treatment, real needles vs. sham. Result: no difference. Effect size 0.06, 95% CI −0.36 to 0.24, p=0.70. Secondary outcomes (dyspareunia, sexual function) also null. Response rates were essentially identical: 58% acupuncture vs. 57% placebo. So the confirmatory test the trial was built around failed.
The only “win” came from Aim 2, a secondary duration analysis: among people who responded, the placebo group relapsed to baseline pain faster (hazard ratio 2.72, 95% CI 1.13–6.54, p=0.017). That single number is the source of the article’s “12 weeks vs. 4 weeks” headline.
Blinding failed in the active arm — and this is the deepest issue. The whole point of the design is to strip out expectation. Bang’s index showed the placebo group was properly blinded (~0, non-significant), but the penetrating-needle group was not: acupuncturists guessed correctly far above chance (index 0.43–0.58, p<0.001) and so did the real-needle patients (0.34–0.35, p<0.01). So in a trial designed to isolate specific effect from placebo, practitioners and patients in the active arm often knew it was real. Differential expectation conveyed nonverbally could produce exactly the durability gap they saw — with zero physiological mechanism. The authors acknowledge this. It’s the single best alternative explanation for the only positive result.
I don’t really want to write this quick take, but omitting this negative result would filter evidences beyond my comfort.
A null effect on pain relief from acupuncture in a pre-registered, improperly double-blinded study (via National Geographic via Facebook)
I didn’t read the paper beyond AI summary, I read the national geographic article in full (which is misleading according to claude)
Selected AI Summary (Full Transcript)
Short version: the underlying paper is methodologically honest and reasonably well-run, but its evidence for acupuncture having a specific (beyond-placebo) effect is weak. The National Geographic article substantially oversells it — it leads with a fragile secondary finding and quietly drops the fact that the trial’s primary outcome was null.
The pre-registered primary outcome was vulvar pain (Average Pain Intensity) at end of treatment, real needles vs. sham. Result: no difference. Effect size 0.06, 95% CI −0.36 to 0.24, p=0.70. Secondary outcomes (dyspareunia, sexual function) also null. Response rates were essentially identical: 58% acupuncture vs. 57% placebo. So the confirmatory test the trial was built around failed.
The only “win” came from Aim 2, a secondary duration analysis: among people who responded, the placebo group relapsed to baseline pain faster (hazard ratio 2.72, 95% CI 1.13–6.54, p=0.017). That single number is the source of the article’s “12 weeks vs. 4 weeks” headline.
Blinding failed in the active arm — and this is the deepest issue. The whole point of the design is to strip out expectation. Bang’s index showed the placebo group was properly blinded (~0, non-significant), but the penetrating-needle group was not: acupuncturists guessed correctly far above chance (index 0.43–0.58, p<0.001) and so did the real-needle patients (0.34–0.35, p<0.01). So in a trial designed to isolate specific effect from placebo, practitioners and patients in the active arm often knew it was real. Differential expectation conveyed nonverbally could produce exactly the durability gap they saw — with zero physiological mechanism. The authors acknowledge this. It’s the single best alternative explanation for the only positive result.
I don’t really want to write this quick take, but omitting this negative result would filter evidences beyond my comfort.