But, if you multiply “once in every ten thousand years” by all the different kinds of things that could be said to happen once every ten thousand years, don’t you get something closer to “many times a day”?
The computation is not relevant, because when you make a prediction that, say, some excursion in the stock market will happen only once in ten thousand years, you are making a prediction about that specific thing, not ten thousand things. It will be a thing you have never seen, because if you had seen it happen, you could not claim it would only happen once in ten thousand years—the observation would be a refutation of that claim. Since you have not seen it, you are deriving it from a theory, and moreover a theory applied at an extreme it has never been tested at. For such a prediction to be reliable, you need to know that your theory actually grasps the basic mechanism of the phenomenon, so that the observations that you have been able to make justify placing confidence in its extremes. This is a very high bar to reach. Here are a few examples of theories where extremes turned out to differ from reality:
Newtonian gravity --> precession of Mercury
Ideal gas laws --> non-ideal gases
Daltonian atomic theory --> multiple isotopes of the same element
The computation is directly relevant, given that Taleb is talking about how often he sees “should only happen every N years” in newspapers and faculty news. Doesn’t he realise how many things newspapers report on? Astronomy faculties are pretty good for this too, since they watch ridiculous numbers of stars at once.
You can’t just ignore the multiple comparisons problem by saying you’re only making a prediction about “one specific thing”. What about all the other predictions about the stock market you made, that you didn’t notice because they turned out to be boringly correct?
Intuition pump: my theory says that the sequence of coinflips HHHTHHTHTT-THHTHHHTT-TTHTHTTTTH-HTTTHTHHHTT, which I just observed, should happen about once every 7 million years.
Intuition pump: my theory says that the sequence of coinflips HHHTHHTHTT-THHTHHHTT-TTHTHTTTTH-HTTTHTHHHTT, which I just observed, should happen about once every 7 million years.
Intuition pump: if I choose an interesting sequence of coinflips in advance, I will never see it actually happen if the coinflips are honest. There aren’t enough interesting sequences of 40 coinflips to ever see one. Most of them look completely random, and in terms of Kolmogorov complexity, most of them are: they cannot be described much more compactly than by just writing them out.
Now, we have a good enough understanding of the dynamics of tossed coins to be fairly confident that only deliberate artifice would produce a sequence of, say, 40 consecutive heads. We do not have such an understanding of the sort of things that appear in the news as “should only happen every N years”.
There aren’t enough interesting sequences of 40 coinflips to ever see one.
Every sequence of 40 coin flips is interesting. Proof: Make a 1 to 1 relation on the sequence of 40 coin flips and a subset of the natural numbers, by making H=1 and T=0 and reading the sequence as a binary representation. Proceed by showing that every natural number is interesting.
The computation is not relevant, because when you make a prediction that, say, some excursion in the stock market will happen only once in ten thousand years, you are making a prediction about that specific thing, not ten thousand things. It will be a thing you have never seen, because if you had seen it happen, you could not claim it would only happen once in ten thousand years—the observation would be a refutation of that claim. Since you have not seen it, you are deriving it from a theory, and moreover a theory applied at an extreme it has never been tested at. For such a prediction to be reliable, you need to know that your theory actually grasps the basic mechanism of the phenomenon, so that the observations that you have been able to make justify placing confidence in its extremes. This is a very high bar to reach. Here are a few examples of theories where extremes turned out to differ from reality:
Newtonian gravity --> precession of Mercury
Ideal gas laws --> non-ideal gases
Daltonian atomic theory --> multiple isotopes of the same element
The computation is directly relevant, given that Taleb is talking about how often he sees “should only happen every N years” in newspapers and faculty news. Doesn’t he realise how many things newspapers report on? Astronomy faculties are pretty good for this too, since they watch ridiculous numbers of stars at once.
You can’t just ignore the multiple comparisons problem by saying you’re only making a prediction about “one specific thing”. What about all the other predictions about the stock market you made, that you didn’t notice because they turned out to be boringly correct?
Intuition pump: my theory says that the sequence of coinflips HHHTHHTHTT-THHTHHHTT-TTHTHTTTTH-HTTTHTHHHTT, which I just observed, should happen about once every 7 million years.
Intuition pump: if I choose an interesting sequence of coinflips in advance, I will never see it actually happen if the coinflips are honest. There aren’t enough interesting sequences of 40 coinflips to ever see one. Most of them look completely random, and in terms of Kolmogorov complexity, most of them are: they cannot be described much more compactly than by just writing them out.
Now, we have a good enough understanding of the dynamics of tossed coins to be fairly confident that only deliberate artifice would produce a sequence of, say, 40 consecutive heads. We do not have such an understanding of the sort of things that appear in the news as “should only happen every N years”.
Feynman on the same theme.
Every sequence of 40 coin flips is interesting. Proof: Make a 1 to 1 relation on the sequence of 40 coin flips and a subset of the natural numbers, by making H=1 and T=0 and reading the sequence as a binary representation. Proceed by showing that every natural number is interesting.