If two things are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
Edit:
If two variables are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
If two things are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
That’s 28 words. Isn’t it a bit long? (Still upvoted because the first sentence stands on its own with just 8 words.)
Consider a sine wave. It can be observed in a great number of phenomena, from the sound produced by a tuning fork to the plot of temperature in mid-latitudes throughout the year. All measurements which produce something resembling a sine wave are correlated. Remember that correlation (well, at least Pearson’s correlation—I assume that’s what is meant here) is invariant to linear transformations so different scale is not a problem.
Correlation isn’t a property of a pair of mathematical functions or a pair of physical systems, it’s a property of a pair of random variables.
“A and B are correlated” means “Observing A can change your probabilistic beliefs about B”.
If you already know that A and B are both sine waves, then neither has any belief-updating power over the others, there’s no randomness in the random variables.
(I know that’s not 100% precise… someone else please improve.)
In the vast majority of cases involving sine waves, the correlation between A and B is due to the common cause of time. Space is also a common cause of such correlations.
However, if you imagine a sine wave in time and another sine wave in space, they have no correlation until you impose a correlation between space and time (e.g., by using a mapping from x to t). In that case, Armok’s comment about a logical rather than physical cause might apply.
I started writing a reply to this comment, but as I was thinking through it I realized that the situation is actually WAY more interesting than I thought and requires a whole post. I’ve posted it in discussion:
I don’t understand this. Which logical fact is the common cause? The fact that the measurements are correlated? Doesn’t the whole thing collapse into a circle, then?
If two things are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
That doesn’t seem to be strictly true. Of all the things that are correlated it would seem that there would be some that have none of the listed causal relationships. It is merely highly probable that one of those is the case.
To the mathematicians, correlation is a statement about random variables, and not the same as empirical correlation (which is a statement about samples, and might be spurious).
Of course the world isn’t made of random variables, but only in the same sense that the world isn’t made of causal models. They are models, and “correlation” and “causation” are features of the model which don’t exist in the real world. In a causal model, correlation implies causation (somewhere).
To the mathematicians, correlation is a statement about random variables
But then this “true correlation” is unobservable, is it not? Except for trivial cases we can never know what it is and can only rely on estimates, aka empirical correlations.
In a causal model, correlation implies causation (somewhere).
Well, that makes Pearl’s statement an uninteresting tautology. Correlation implies causation because we construct models this way...
Emphasizing random variables sounds pretty frequentist to me, while the source being summarized is bayesian. But, yes, models are made of random variables.
thanks, this is exactly the case. a better objection is, it’s not strictly true because things can be some complex net of the above cases, and it doesn’t always break down into one of the four, but that doesn’t fit in “15” words, and it’s less important
edit: also it’s possible in rare cases for things to be uncorrelated but causally connected
To address your correct criticism, how about we modify apophenia’s “15” words to:
• If two things are reliably correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
A 15-word version is possible but awkward:
• Reliable correlation implies causation: one causes the other, or there’s common cause, or common effect.
Potentially a great deal of complexity is smuggled into the word “reliable”.
--
Edit: A friend pointed out to me that the above sentences provide unbalanced guidance for intuitions. A more evenly balanced version is:
• Reliable correlation implies causation and unreliable correlation does not.
It goes against the spirit of “15 words” to insist on strict truth.
I would suggest that it goes against the spirit of Judea Pearl’s Causality to say things that are false or misleading.
Do note that I actually support the example, despite the problems. I expect that the surrounding context in Pearl’s work more than adequately explains the relevant details. What I would object to is any attempt to suppress discussion of the limitations of such claims—so if it was the case that the “spirit of ’15 words’” discourages discussion and clarification then I would reject it as inappropriate on this site.
“15 words” is a secretly a verb rather than a noun. I definitely think discussion and clarification is good, although in this particular thread I’m sad to some people engaging solely in that and missing an opportunity to try out the exercise instead.
“15 words” is a secretly a verb rather than a noun.
As the thread creator you are entitled to specify the way you want the phrase to be used and what sort of replies you want. That said, it seems that the norms that you are attempting to create and enforce for this ’15 words’ activity don’t belong on this site. It seems to amount to provoking and enforcing all the worst of the failures of critical thought that constantly crop up in the “Rationality” Quotes threads. Given as a premise that I hold that belief you could infer that my voting policy must be to downvote:
Any thread or comment requesting the ‘action’ “15 words” be performed.
Any attempt to criticise, suppress or dismiss clarifications, elaborations and analysis that crop up in response to quotes.
Any comment, regardless of overall merit, for which a minor clarification is necessary but would be prohibited or discouraged. Note that this applies to the ancestral quote by Pearl which I had previously upvoted. In a context of enforced uncriticality any deviation from accuracy becomes a critical failure.
I’m sad to some people engaging solely in that and missing an opportunity to try out the exercise instead.
That isn’t what you saw. You saw people engaging in that in addition to engaging with the the exercise. They lost no opportunity, you merely couldn’t tolerate the critical engagement that is an integral part of discussion on a rationalist forum.
It’s possible to find “spurious” correlations in a limited data sample, if two things just “happen” to happen together often by chance. But I don’t think that really counts. Did you have any other scenarios in mind?
It’s possible to find “spurious” correlations in a limited data sample, if two things just “happen” to happen together often by chance. But I don’t think that really counts.
When absolute claims are made with exhaustive lists of possibilities then things can “not count” only when excluded explicitly. When dealing with things at the level of precision and rigour that Pearl works at the difference between ‘almost true’ and ‘true’ matters. Even with the (‘probably’ or ‘overwhelmingly likely’) caveat in place the statement remains valuable. It is still worth including such a parenthetical so as to avoid confusion.
Did you have any other scenarios in mind?
No, the set of all correlations that are not causally related in one of the listed ways seems to fit the criteria “limited” and to whatever extent they can be described as ‘spurious’ that description would apply to all of them. Admittedly, some of them are ‘limited’ only by such things as the size of the universe but the larger the sample the higher the improbability.
I would replace ‘spurious’ with ‘misleading’. A correlation just is. There isn’t anything ‘fake’ or ‘invalid’ about it. The only thing that could be wrong about it is using it to draw an incorrect conclusion.
I have a feeling including a parenthetical like that would invite more confusion than it avoids. “Oh cool, I guess my magical ESP powers are just one of the unlikely cases where I can be correlated with the hidden coin flips without any causal influence.”
Because “correlation” is normally taken to mean a systematic effect that can be expected to be predictive of future samples, or something. In this specific case, Pearl probably means something more precise by it (like correlations between nodes in a particular causal model).
I suppose you could accurately clarify the original quote by saying “systematic correlation”, which would pin down the idea referred to for people who haven’t read the book.
I have a feeling including a parenthetical like that would invite more confusion than it avoids. “Oh cool, I guess my magical ESP powers are just one of the unlikely cases where I can be correlated with the hidden coin flips without any causal influence.”
The unqualified version is more compatible with muddled thinking about ESP than the qualified version. Specifically, it outright excludes the possibility “No, you were just lucky” from consideration.
In this specific case, Pearl probably means something more precise by it (like correlations between nodes in a particular causal model).
With enough data from the two correlands, this goes away. I don’t know the exact math, but I think there’s a way to say the number of variables you’re looking at, and the strength of a given correlation, and get a probability that it’s really there.
This goes away only in the limit as the sample size goes to infinity.
For a finite sample size (and given a certain set of assumptions) you can establish a range of values within which you believe “true” correlation resides, but this range will never contract to a single point.
I think the problem may be what counts as correlated. If I toss two coins and both get heads, that’s probably coincidence. If I toss two coins N times and get HH TT HH HH HH TT HH HH HH HH TT HH HH HH HH HH TT HH TT TT HH then there’s probably a common cause of some sort.
But real life is littered with things that look sort of correlated, like price of X and price of Y both (a) go up over time and (b) shoot up temporarily when the roads are closed, but are not otherwise correlated, and it’s not clear when this should apply (even though I agree it’s a good principle).
Judea Pearl, Causality:
If two things are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
Edit: If two variables are correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
That’s 28 words. Isn’t it a bit long? (Still upvoted because the first sentence stands on its own with just 8 words.)
http://xkcd.com/882/
Sometimes the cause is you’ve been looking at too many random data sets.
I am confused, that doesn’t seem to be true.
Consider a sine wave. It can be observed in a great number of phenomena, from the sound produced by a tuning fork to the plot of temperature in mid-latitudes throughout the year. All measurements which produce something resembling a sine wave are correlated. Remember that correlation (well, at least Pearson’s correlation—I assume that’s what is meant here) is invariant to linear transformations so different scale is not a problem.
Correlation isn’t a property of a pair of mathematical functions or a pair of physical systems, it’s a property of a pair of random variables.
“A and B are correlated” means “Observing A can change your probabilistic beliefs about B”.
If you already know that A and B are both sine waves, then neither has any belief-updating power over the others, there’s no randomness in the random variables.
(I know that’s not 100% precise… someone else please improve.)
In the vast majority of cases involving sine waves, the correlation between A and B is due to the common cause of time. Space is also a common cause of such correlations.
However, if you imagine a sine wave in time and another sine wave in space, they have no correlation until you impose a correlation between space and time (e.g., by using a mapping from x to t). In that case, Armok’s comment about a logical rather than physical cause might apply.
I don’t understand what does that mean. In which sense can time be thought of as a cause?
I started writing a reply to this comment, but as I was thinking through it I realized that the situation is actually WAY more interesting than I thought and requires a whole post. I’ve posted it in discussion:
http://lesswrong.com/r/discussion/lw/is7/the_cause_of_time/
Sorry if it’s a bit unclear right now, hopefully I’ll have time to add some diagrams this weekend.
This is a case of a common cause, in the form of a logical fact rather than a physical one.
I don’t understand this. Which logical fact is the common cause? The fact that the measurements are correlated? Doesn’t the whole thing collapse into a circle, then?
The fact of the shape of a sine curve.
Only if the frequencies are identical. In that case, follow the improbability and ask how they come to be identical.
That doesn’t seem to be strictly true. Of all the things that are correlated it would seem that there would be some that have none of the listed causal relationships. It is merely highly probable that one of those is the case.
To the mathematicians, correlation is a statement about random variables, and not the same as empirical correlation (which is a statement about samples, and might be spurious).
Of course the world isn’t made of random variables, but only in the same sense that the world isn’t made of causal models. They are models, and “correlation” and “causation” are features of the model which don’t exist in the real world. In a causal model, correlation implies causation (somewhere).
But then this “true correlation” is unobservable, is it not? Except for trivial cases we can never know what it is and can only rely on estimates, aka empirical correlations.
Well, that makes Pearl’s statement an uninteresting tautology. Correlation implies causation because we construct models this way...
Emphasizing random variables sounds pretty frequentist to me, while the source being summarized is bayesian. But, yes, models are made of random variables.
thanks, this is exactly the case. a better objection is, it’s not strictly true because things can be some complex net of the above cases, and it doesn’t always break down into one of the four, but that doesn’t fit in “15” words, and it’s less important
edit: also it’s possible in rare cases for things to be uncorrelated but causally connected
To address your correct criticism, how about we modify apophenia’s “15” words to:
• If two things are reliably correlated, there is causation. Either A causes B, B causes A, they have common cause, or they have a common effect you’re conditioning on.
A 15-word version is possible but awkward:
• Reliable correlation implies causation: one causes the other, or there’s common cause, or common effect.
Potentially a great deal of complexity is smuggled into the word “reliable”.
--
Edit: A friend pointed out to me that the above sentences provide unbalanced guidance for intuitions. A more evenly balanced version is:
• Reliable correlation implies causation and unreliable correlation does not.
It goes against the spirit of “15 words” to insist on strict truth. The merit of the quote lies in the fourth clause.
That’s the big surprise. The point of boiling it down to “15 words” is to pick which subtlety makes it into the shortest formulation.
I would suggest that it goes against the spirit of Judea Pearl’s Causality to say things that are false or misleading.
Do note that I actually support the example, despite the problems. I expect that the surrounding context in Pearl’s work more than adequately explains the relevant details. What I would object to is any attempt to suppress discussion of the limitations of such claims—so if it was the case that the “spirit of ’15 words’” discourages discussion and clarification then I would reject it as inappropriate on this site.
“15 words” is a secretly a verb rather than a noun. I definitely think discussion and clarification is good, although in this particular thread I’m sad to some people engaging solely in that and missing an opportunity to try out the exercise instead.
As the thread creator you are entitled to specify the way you want the phrase to be used and what sort of replies you want. That said, it seems that the norms that you are attempting to create and enforce for this ’15 words’ activity don’t belong on this site. It seems to amount to provoking and enforcing all the worst of the failures of critical thought that constantly crop up in the “Rationality” Quotes threads. Given as a premise that I hold that belief you could infer that my voting policy must be to downvote:
Any thread or comment requesting the ‘action’ “15 words” be performed.
Any attempt to criticise, suppress or dismiss clarifications, elaborations and analysis that crop up in response to quotes.
Any comment, regardless of overall merit, for which a minor clarification is necessary but would be prohibited or discouraged. Note that this applies to the ancestral quote by Pearl which I had previously upvoted. In a context of enforced uncriticality any deviation from accuracy becomes a critical failure.
That isn’t what you saw. You saw people engaging in that in addition to engaging with the the exercise. They lost no opportunity, you merely couldn’t tolerate the critical engagement that is an integral part of discussion on a rationalist forum.
It’s possible to find “spurious” correlations in a limited data sample, if two things just “happen” to happen together often by chance. But I don’t think that really counts. Did you have any other scenarios in mind?
When absolute claims are made with exhaustive lists of possibilities then things can “not count” only when excluded explicitly. When dealing with things at the level of precision and rigour that Pearl works at the difference between ‘almost true’ and ‘true’ matters. Even with the (‘probably’ or ‘overwhelmingly likely’) caveat in place the statement remains valuable. It is still worth including such a parenthetical so as to avoid confusion.
No, the set of all correlations that are not causally related in one of the listed ways seems to fit the criteria “limited” and to whatever extent they can be described as ‘spurious’ that description would apply to all of them. Admittedly, some of them are ‘limited’ only by such things as the size of the universe but the larger the sample the higher the improbability.
I would replace ‘spurious’ with ‘misleading’. A correlation just is. There isn’t anything ‘fake’ or ‘invalid’ about it. The only thing that could be wrong about it is using it to draw an incorrect conclusion.
I have a feeling including a parenthetical like that would invite more confusion than it avoids. “Oh cool, I guess my magical ESP powers are just one of the unlikely cases where I can be correlated with the hidden coin flips without any causal influence.”
Because “correlation” is normally taken to mean a systematic effect that can be expected to be predictive of future samples, or something. In this specific case, Pearl probably means something more precise by it (like correlations between nodes in a particular causal model).
I suppose you could accurately clarify the original quote by saying “systematic correlation”, which would pin down the idea referred to for people who haven’t read the book.
The unqualified version is more compatible with muddled thinking about ESP than the qualified version. Specifically, it outright excludes the possibility “No, you were just lucky” from consideration.
This exception applies in that case.
Doesn’t count?!
With enough data from the two correlands, this goes away. I don’t know the exact math, but I think there’s a way to say the number of variables you’re looking at, and the strength of a given correlation, and get a probability that it’s really there.
This goes away only in the limit as the sample size goes to infinity.
For a finite sample size (and given a certain set of assumptions) you can establish a range of values within which you believe “true” correlation resides, but this range will never contract to a single point.
I think the problem may be what counts as correlated. If I toss two coins and both get heads, that’s probably coincidence. If I toss two coins N times and get HH TT HH HH HH TT HH HH HH HH TT HH HH HH HH HH TT HH TT TT HH then there’s probably a common cause of some sort.
But real life is littered with things that look sort of correlated, like price of X and price of Y both (a) go up over time and (b) shoot up temporarily when the roads are closed, but are not otherwise correlated, and it’s not clear when this should apply (even though I agree it’s a good principle).
An alternative version which avoids most of the complaints in replies below:
Correlation doesn’t imply causation, but it’s damn strong evidence!
(Please reply if you remember either the exact wording or the source of that quote).
Note, as I discuss here for this to be true you need to allow mathematical truths (and the laws of physics) to serve as causes.