I apologize; I made an error in my original comment. I was actually referring to high blood pressure rather than diabetes. 15 out of the 26 people in the control group had high blood pressure, which is greater than the number of people who needed ICU care. Using your (maximally generous) assumptions, we would have zero non-hypertensive patients from either group needing ICU care.
Firstly, two risk factors were more common among the treatment groups: <60 years of age, immunosuppressed & transplanted.
Absolutely true, but the overall risk factor prevalence was still significantly higher in the control group. Furthermore, I’m not sure if all risk factors are created equal. Regardless, the overall point is that the two groups had significant differences in important characteristics.
*In order to achieve a p<.05 the lack of blinding/fuzziness would must have failed to send 16 of the 46 treatment group members to the ICU.* That is still not likely without deliberate fraud.
I think it’s more likely that they sent a few of the control group members to the ICU unnecessarily. If you figure that the difference in risk factors between the two groups accounts for a couple of the extra ICU cases, the placebo effect accounts for another couple, and unnecessary ICU admission accounts for another couple, it brings the P-value up pretty dramatically. I’m not statistically literate enough to know how to properly adjust for those factors and get an exact number, but it doesn’t seem to require deliberate fraud.
Just to be clear, I still think that there is probably at least some sort of real effect here. I’m just advocating caution in interpreting the results of a tiny study with clear flaws. I don’t really understand why there was no placebo control or double blinding, and that makes me more suspicious that there are other flaws that I’m not educated enough to notice. For example, the way that they describe the ICU admission criteria suggests that the presence of a comorbidity is itself a factor for ICU admission. If that’s the case, the differences in the risk factor numbers become even more important.
Regarding the Vitamin D study, it doesn’t seem like it was placebo controlled. Given the lack of placebo control, I’m not sure how it could be double blind. There were also a number of risk factors where the treatment and control groups had significant differences, most notably diabetes (present in 2.5x as many patients in the control group). If you combine the differences in the characteristics of the groups with the lack of placebo control and blinding and the “fuzziness” of ICU admission criteria, that could start to explain the effect without any deliberate fraud.
Of course, it’s still an impressive enough result that we absolutely need to be doing further research. I just think we should be cautious about how much weight we put on a single tiny study with unclear controls and blinding.