Thanks for a detailed reply!
p-values and priors do not mix
I guess this is what I get for my statistics-knowledge being an odd mongrel mix of Bayesian vibes gathered from LessWrong and frequentist stats from biology grad school. You seem much more knowledgeable on the formalizations so I trust you’re right that they can’t formally mix, but informally to me it seems like there must be some way to mix them—fundamentally, this is just “extraordinary claims require extraordinary evidence”. To use an example from my day job: if I’m testing whether knocking out a candidate gene that I think will increase tryptophan in a plant tissue does indeed increase the tryptophan, p < 0.05 feels like a fine cutoff. If I’m testing whether that same KO causes my plants to communicate with me telepathically, I’d be crazy to tell anyone about my results unless I was seeing p < 0.001 in at least two independent experiments.
The difference in required p-value threshold to me seems to come down to the prior. Perhaps there’s no formal framework that combines them, but empirically I think that’s what I’m doing.
The mathematics of causality is still not part of a standard stats curriculum
.
You seem to be correct here, but it strikes me as strange because quantifying the evidence for causality was one of the central themes of most of my stats classes (with names like “intro statistics” and “statistical design of experiments”).
Possibly the synthesis here is that most of what non-math people learn doesn’t qualify as real math—I only took the super basic stuff that doesn’t get into the actual math of causality (despite minoring in math in undergrad and taking more math than most in a bio PhD).
I would like there to be a formal way to render both DNA and dominoes as a short causal chain. But I don’t have one.
Thinking about it a bit more, I think I’m happy to bite the bullet here and say the causal chain is long (though well-approximated by both of our very short descriptions), and that this is one of the exceptions to the point in footnote 4 that real-world correlations are generally well below 1. The causal chain is quite long, but DNA replication is extremely good—something like 10^-7 errors per base of DNA per generation—pretty dang close to 1 (my guess is domino setups by hobbyists also have failure rates under 10^-3). I’m sure we could find a handful more examples of things with long chains of very high correlation, but not that many—pretty few in biology are over 0.95.
A classic example from Pearl’s 2009 book
Thanks, that was fun to work out, and appropriately simple for a humble biologist. In the 2x2 matrix of “corr or not” and “causal linkage or not”, this is the opposite square to the one I’m looking for—causal but not correlated. I agree that such things happen (they happen a lot in biology due to regulatory feedback loops), and I now see that this is indeed a non-faithful network.
Is there a similar toy example for “corr but no causal linkage”, e.g. “two variables on opposite sides of the program whose value is correlated for no good reason”? I spent a while asking Claude Opus 4.6 for one and it couldn’t come up with any (it came up with a toy where two variables would always be the same constant, but correlation there is undefined so I don’t count it).
Thank you for providing counterexamples! Quite useful. and convinces me that the broadest version of this is false. And apologies for the slow reply—the linked papers took a while to absorb (as you may guess, I’m not a mathematician).
1. I tried for a while, but couldn’t properly follow this paper. From reading the abstract, goung through the figures, your summary, and discussing it with Claude (Opus 4.6), I think the core issue here is the same problem as point 2 - the measures are not independent. My wife’s phrasing here is that “there’s a causal link between position at time t and position at time t+1” (in this case + noise, in the case of point 2 plus velocity times time between measurements).
2. Thank you—slightly embarrassed I didn’t find this one myself. I think the issue here is due to repeated measurement—the moon-earth distance now and at t+1s are obviously connected. I think the missing criteria here is independence—repeatedly measuring the same thing to see evolution over time breaks it. This requires some narrowing of the core thesis—my proposed added text is “I’m talking about correlations that survive proper statistical scrutiny—where the p-value is computed correctly for the data structure at hand. A naive Pearson correlation between two autocorrelated time series, or two monotonic trends, can look impressive while meaning nothing, but that’s not a real correlation any more than a loaded die produces a real test of probability. The well-known tools for handling this (differencing, detrending, cointegration tests, correcting for effective sample size) exist precisely because statisticians already understand that autocorrelation inflates apparent correlation. My claim applies to correlations that remain after you’ve done the statistics right.”
I think the claim “correlation of independent measures implies causation” is still interesting and surprising (probably not to this crowd who point out plenty of more rigorous prior work, but to most biologists at least), though a bit less exciting than my original claim. In particular, it’s less exciting because it doesn’t have a bulletproof checklist for “doing the statistics right”, which may or may not be possible to make.
3. This paper I could understand. I’ve worked through all their examples, and don’t think any are counterexamples to my point. Indeed causation can exist without correlation (as you say, very common in biology and technology), I’ve never claimed otherwise. The paper shows some examples where correlation is high despite the causal path containing intermediates with lower correlation which is intriguing, but I believe every correlation in the paper is explained by a causal path that links the two things correlated. As you say, not necessarily a direct causal connection, but still a “relatively short causal chain linking those things”—an indirect causal connection.
My favorite quote from the paper was “the simple maxim that ‘correlation does not imply causation’ having been superceded by methods such as those set out in [9, 14], and in shorter form in [10]”. Good to see that others (Pearl in addition to Reichenbach) have made something like the point I’m aiming to make here, as well as the restriction from time-series data that you brought to my attention in (2) - “cited limit attention to systems whose causal connections form a directed acyclic graph, together with certain further restrictions, and also do not consider dynamical systems or time series data.”
So far I don’t think cyclic systems cause any correlations between causally-unlinked variables, but the fact that people doing this formally haven’t solved it makes me hesitant to make any claims as an outsider to the field.