An experiment
Alexei’s postmortem of Arbital got me thinking about incentivizing good online explanations, so I got an idea for a test that we can run here and now:
What’s your simplest, most vivid, most memorable explanation of correlation does not equal causation for someone who’s unaware of the concept? In your own words, no linking.
This is probably not the “simplest” explanation I could give, but I find it memorable. I heard this story in college, but haven’t looked for the reference.
~~~
There was apparently once a study for a new medication, which the researchers really thought would work. So they went through the whole process of a randomized trial, and it came back with no result. The medicine didn’t help.
They thought about this for a while, and then wondered if maybe the problem was that people weren’t taking the medicine regularly enough. Perhaps people were skipping days, or taking it at different times every day, and this was screwing things up? So they re-ran the study, this time tracking compliance. And lo and behold, people who regularly took the medicine did indeed do better than average!
As good scientists, however, they applied the same analysis to their placebo tests. And they found that people who regularly took the placebo also did better than average.
It turns out that the medicine was indeed worthless, but that the sort of people who remember to take their medicine every day are just generally healthier, because they are more likely to be concientious and do things like exercise and eat well.
Wow, that’s really good! It was worth making the post just to hear that story.
My first stats teacher used the example of juvenile delinquency and ice cream sales, which rise and fall together. She gave us a few minutes to try and explain why this might be before pointing out that both of these rise dramatically during the summer, when it’s hot out and kids are out of school.
Haha, I often ask people “did you know taller kids are better at math?” Then wait a few seconds and say “because they’re older”.
Here’s mine, just to get things started:
Malaria is statistically linked with getting bitten by a mosquito, feeling ill, and going to a hospital. The first of these causes malaria, the second is caused by malaria, and the third cures malaria. So when you read in the news that something is statistically linked with something else, that means nothing until you prove a causal story.
Buying expensive things does not make you rich.
I’ve stumbled on a slightly offtopic but amazing example by user “cortesoft” on HN:
There is a whole book of spurious correlations.
However I question whether a lack of memorable explanations is a real problem with our world. I personally don’t feel a need for more explanations of trivial things. Aren’t there enough already? If we did have Arbital with explanations like that, who would be the audience? And what would be the intended effect? People who don’t want to know will not read it, and people who want to know should have figured it out already.
Well, this experiment was about supply, not demand. Would people want to show off cool explanations to each other, without students? It seems like no. That suggests some follow-up experiments. You can also do experiments to test demand, they would have to be different though.
Someone takes a class. They take a test, which they have to pass to pass the class. They pass the test and the class.
How do we know they couldn’t have passed the test before taking the class?
Alternatively, from xkcd (from memory):
Person 1: I used to think correlation implied causation.
Person 1: Then I took a stats class.
Person 1: Now I don’t.
Person 2: So it helped.
Person 1: Maybe.
Hover text: correlation doesn’t point to causation, but it inclines its head and raises its eyebrows.