Probably the easiest way to talk about it is using standardized effect sizes, i.e. multiplying and dividing the effects with the population standard deviations to end up with unitless numbers.
If you’ve got a causal chain, e.g. SNP → alcohol consumption → IQ, and this causal chain is the only thing linking the variables together (no confounding or alternative mechanisms or reverse causation), then the correlation between variables in the chain is computed by multiplying the effects together along the chain. For instance, if we let r(X, Y) denote the correlation between X and Y, and e(X, Y) denote the standardized effect of X on Y, then we have:
Now, the basic principle of Mendelian randomization is that under appropriate circumstances, r(SNP, X) = e(SNP, X). So we can rearrange the equation above to get:
This is essentially how the MR (and other IV) effect size estimates are computed (though they may use fancier math for various reasons).
Now, the first thing that should make you uncomfortable here is that you are dividing by a very small number, r(SNP, alcohol consumption). But what effects could that have? Well, let us for a moment imagine that the effect actually goes in the opposite direction, so SNP → IQ → alcohol consumption (possibly unlikely in this case for mechanistic reasons, but the point is that one should always discuss those mechanistic reasons). Then the previous estimate reduces to:
Most effect sizes in social science tend to be fairly small, so e(IQ, alcohol consumption) is probably also small, and therefore 1/e(IQ, alcohol consumption) is probably fairly large. For instance, if e(IQ, alcohol consumption) was 0.1, then your estimated effect would be 10 (in practice standardized effect sizes are nearly always between −1 and 1, because otherwise you need strong alternative effects that cancel them out, so this is “impossibly big”—but there are other biases that could lead to milder results than “impossibly big”).
(The full rules for how to compute these sorts of things, including for more unstandardized effect sizes and more complex causal graphs, are called path tracing rules/path analysis.)
If I understand your reply correctly, your conclusion is that epidemiologists should:
Discuss the mechanistic reasons re: the direction of IQ <-> Alcohol consumption and
Especially distrust mendelian randomization studies where (1.) isn’t strongly argued for, and which get really large estimates.
I think these are important points!
Yep!
But given the very small effects estimated here, you aren’t arguing for a change to the interpretation of the studies in the post, right? :-)
In principle, this sort of problem can also lead to smaller effects, but it’s probably less likely, so it might be fine? I mean it’s possible I’ve missed something that could make it a problem.
Looking closer, the ADH1B that the variant is in is involved in alcohol metabolism, so that probably gives a plausible idea of how the MR mechanism would work.
Discuss the mechanistic reasons re: the direction of IQ <-> Alcohol consumption and
Actually I wanna issue a correction here. They should discuss the mechanistic reasons for the SNP → alcohol consumption link. The point of MR is to figure out things about the IQ <-> alcohol consumption causal link without necessarily having good mechanistic knowledge of it. But as Pearl says, if you put no causes in, you get no causes out; in exchange for the IQ <-> alcohol consumption causal link, you must know even more details about the effects of the SNP that you use for estimation.
Can you expand on this?
Sure.
Probably the easiest way to talk about it is using standardized effect sizes, i.e. multiplying and dividing the effects with the population standard deviations to end up with unitless numbers.
If you’ve got a causal chain, e.g. SNP → alcohol consumption → IQ, and this causal chain is the only thing linking the variables together (no confounding or alternative mechanisms or reverse causation), then the correlation between variables in the chain is computed by multiplying the effects together along the chain. For instance, if we let r(X, Y) denote the correlation between X and Y, and e(X, Y) denote the standardized effect of X on Y, then we have:
r(SNP, IQ) = e(SNP, alcohol consumption) * e(alcohol consumption, IQ)
Now, the basic principle of Mendelian randomization is that under appropriate circumstances, r(SNP, X) = e(SNP, X). So we can rearrange the equation above to get:
e(alcohol consumption, IQ) = r(SNP, IQ)/r(SNP, alcohol consumption)
This is essentially how the MR (and other IV) effect size estimates are computed (though they may use fancier math for various reasons).
Now, the first thing that should make you uncomfortable here is that you are dividing by a very small number, r(SNP, alcohol consumption). But what effects could that have? Well, let us for a moment imagine that the effect actually goes in the opposite direction, so SNP → IQ → alcohol consumption (possibly unlikely in this case for mechanistic reasons, but the point is that one should always discuss those mechanistic reasons). Then the previous estimate reduces to:
r(SNP, IQ)/r(SNP, alcohol consumption) = e(SNP, IQ)/(e(SNP, IQ)*e(IQ, alcohol consumption)) = 1/e(IQ, alcohol consumption)
Most effect sizes in social science tend to be fairly small, so e(IQ, alcohol consumption) is probably also small, and therefore 1/e(IQ, alcohol consumption) is probably fairly large. For instance, if e(IQ, alcohol consumption) was 0.1, then your estimated effect would be 10 (in practice standardized effect sizes are nearly always between −1 and 1, because otherwise you need strong alternative effects that cancel them out, so this is “impossibly big”—but there are other biases that could lead to milder results than “impossibly big”).
(The full rules for how to compute these sorts of things, including for more unstandardized effect sizes and more complex causal graphs, are called path tracing rules/path analysis.)
Martin here, the main author of the above. Thanks a ton for this!
If I understand your reply correctly, your conclusion is that epidemiologists should:
Discuss the mechanistic reasons re: the direction of IQ <-> Alcohol consumption and
Especially distrust mendelian randomization studies where (1.) isn’t strongly argued for, and which get really large estimates.
I think these are important points!
But given the very small effects estimated here, you aren’t arguing for a change to the interpretation of the studies in the post, right? :-)
Yep!
In principle, this sort of problem can also lead to smaller effects, but it’s probably less likely, so it might be fine? I mean it’s possible I’ve missed something that could make it a problem.
Looking closer, the ADH1B that the variant is in is involved in alcohol metabolism, so that probably gives a plausible idea of how the MR mechanism would work.
Actually I wanna issue a correction here. They should discuss the mechanistic reasons for the SNP → alcohol consumption link. The point of MR is to figure out things about the IQ <-> alcohol consumption causal link without necessarily having good mechanistic knowledge of it. But as Pearl says, if you put no causes in, you get no causes out; in exchange for the IQ <-> alcohol consumption causal link, you must know even more details about the effects of the SNP that you use for estimation.