This analysis of the data leads to a seeming contradiction. Within a given region, increased mask usage is correlated with lower growth rates (the 25% claimed effectiveness), but when comparing across regions masks seem to be ineffective. Depending on the causal story you want to tell, either of these claims could be true.
It is possible that people wear masks most in places where COVID is most transmissible. This would explain why masks don’t appear effective when comparing across regions.
However it is also possible that the same factors causes both mask wearing to increase and transmissibility to decrease. For instance, if people wear masks in response to an observed spike in cases, then the population immunity caused by the spike will make masks appear to be effective even if are not.
This is an excellent element of this post, and I am very pleased to see it! These two possibilities are actually testable in a model that statistically controls (i.e., includes) transmissibility. Notice that in the first possibility, mask wearing is a (partial) mediator for the effect of transmissibility on cases. In the second, mask wearing and transmissibility are commonly-caused by something else(s). Either way, controlling the transmissibility path improves our estimate of the mask wearing path. [Aside: what if your statistical control was “insufficient” and mask wearing still correlates with the error term in your model? Then your causal estimate loses some causal interpretability, but this critique is virtually impossible to completely address short of randomized experimentation. Still, you try your best and transmissibility here is the biggie that should account not just for itself but also soak up any common causes you might expect to be in the error term.]
This is why beginning-of-period transmissibility factors are included in the model to estimate end-of-period transmissibility results. It’s possible that mask wearing has a negative effect on cases but that it’s more than offset by the big positive direct effect of transmissibility on cases. If you’re interested in the correlation between mask wearing and cases, you’ll be disappointed by the net positive effect, which is of course not interpretable as causal. If you’re interested in the causal effect of mask wearing on cases, you’ll be encouraged by the negative effect and hope to find ways to increase mask wearing besides the epidemic just getting so much worse that people start to wear masks. This is the first possibility. This is also how the second possibility looks statistically—we can still get our estimate of the effect of mask wearing, which ultimately is the focus of the investigation. But whether the first or second possibility is “true” may be relevant not so much for estimating the effect of mask wearing (it’s statistically the same—control for transmissibility), but for getting a wider understanding of the world (does transmissibility cause mask wearing or does something else cause both?, which is not the focus of the study).
I also want to note that the endogeneity critiques of observational (vs. experimental) methods are legit, but there is a lot that can be (and is) done to draw “more causal” conclusions from observational data than mere correlation, and experiments can have their own internal validity concerns, so both approaches are useful for learning about the world.
This is an excellent element of this post, and I am very pleased to see it! These two possibilities are actually testable in a model that statistically controls (i.e., includes) transmissibility. Notice that in the first possibility, mask wearing is a (partial) mediator for the effect of transmissibility on cases. In the second, mask wearing and transmissibility are commonly-caused by something else(s). Either way, controlling the transmissibility path improves our estimate of the mask wearing path.
[Aside: what if your statistical control was “insufficient” and mask wearing still correlates with the error term in your model? Then your causal estimate loses some causal interpretability, but this critique is virtually impossible to completely address short of randomized experimentation. Still, you try your best and transmissibility here is the biggie that should account not just for itself but also soak up any common causes you might expect to be in the error term.]
This is why beginning-of-period transmissibility factors are included in the model to estimate end-of-period transmissibility results. It’s possible that mask wearing has a negative effect on cases but that it’s more than offset by the big positive direct effect of transmissibility on cases. If you’re interested in the correlation between mask wearing and cases, you’ll be disappointed by the net positive effect, which is of course not interpretable as causal. If you’re interested in the causal effect of mask wearing on cases, you’ll be encouraged by the negative effect and hope to find ways to increase mask wearing besides the epidemic just getting so much worse that people start to wear masks. This is the first possibility. This is also how the second possibility looks statistically—we can still get our estimate of the effect of mask wearing, which ultimately is the focus of the investigation. But whether the first or second possibility is “true” may be relevant not so much for estimating the effect of mask wearing (it’s statistically the same—control for transmissibility), but for getting a wider understanding of the world (does transmissibility cause mask wearing or does something else cause both?, which is not the focus of the study).
I also want to note that the endogeneity critiques of observational (vs. experimental) methods are legit, but there is a lot that can be (and is) done to draw “more causal” conclusions from observational data than mere correlation, and experiments can have their own internal validity concerns, so both approaches are useful for learning about the world.