I concur. This has nothing to do with the relevance or value of probability and statistics; it’s just debunking the idea that a correlation coefficient that’s substantial but not very close to +-1 gives you much predictive power.
What makes Simplicio’s performance worse than Salviati’s isn’t the fact that he’s using probability and statistics. It’s the fact that the information he has available is very poor. Describing what he’s got in terms of correlation coefficients has, at most, the effect of obscuring just how terrible they are, but that’s not a problem with probability and statistics, it’s a problem with not understanding probability and statistics.
It certainly is true that a correlation of .6 doesn’t give you a good measurement. (Is that the point?)
That is part of it.
gjm:
I concur. This has nothing to do with the relevance or value of probability and statistics; it’s just debunking the idea that a correlation coefficient that’s substantial but not very close to +-1 gives you much predictive power.
That is more of it.
gjm:
What makes Simplicio’s performance worse than Salviati’s isn’t the fact that he’s using probability and statistics. It’s the fact that the information he has available is very poor.
And this is the final part. As a matter of practical fact—look at almost any scientific paper that presents correlation coefficients—if you are calculating correlations, 0.6 is about typical of the correlations you will be finding, and I think I’m being generous there. The reason you don’t see correlations of 0.995 reported, let alone 0.99995 (i.e. a measurement to two significant figures) is that if your data were that good, you wouldn’t waste your time doing statistics on them. A correlation of 0.6 means that you have poor data and almost no predictive capacity. It takes a correlation of 0.866 to get even 1 bit of mutual information. How often do you see correlations of that size reported?
Statistics is the science of precisely wringing what little information there is from foggy data. And yet, people keep on drawing lines through scatterplots and summarising results as “X’s are Y’s”, even when the implied prediction does only fractionally better than chance.
Eliezer wrote: “Let the winds of evidence blow you about as though you are a leaf, with no direction of your own”, which is very inspiring, but in practical terms cannot be taken literally. If you are being blown up and down the probability scale, your probabilities are nowhere near 0 or 1. You can only be easily swayed when you are ignorant. You can only remain easily swayed by remaining ignorant. The moment you acquire knowledge, instead of precisely measured ignorance, you are wearing lead-weighted boots.
I concur. This has nothing to do with the relevance or value of probability and statistics; it’s just debunking the idea that a correlation coefficient that’s substantial but not very close to +-1 gives you much predictive power.
What makes Simplicio’s performance worse than Salviati’s isn’t the fact that he’s using probability and statistics. It’s the fact that the information he has available is very poor. Describing what he’s got in terms of correlation coefficients has, at most, the effect of obscuring just how terrible they are, but that’s not a problem with probability and statistics, it’s a problem with not understanding probability and statistics.
Douglas_Knight:
That is part of it.
gjm:
That is more of it.
gjm:
And this is the final part. As a matter of practical fact—look at almost any scientific paper that presents correlation coefficients—if you are calculating correlations, 0.6 is about typical of the correlations you will be finding, and I think I’m being generous there. The reason you don’t see correlations of 0.995 reported, let alone 0.99995 (i.e. a measurement to two significant figures) is that if your data were that good, you wouldn’t waste your time doing statistics on them. A correlation of 0.6 means that you have poor data and almost no predictive capacity. It takes a correlation of 0.866 to get even 1 bit of mutual information. How often do you see correlations of that size reported?
Statistics is the science of precisely wringing what little information there is from foggy data. And yet, people keep on drawing lines through scatterplots and summarising results as “X’s are Y’s”, even when the implied prediction does only fractionally better than chance.
Eliezer wrote: “Let the winds of evidence blow you about as though you are a leaf, with no direction of your own”, which is very inspiring, but in practical terms cannot be taken literally. If you are being blown up and down the probability scale, your probabilities are nowhere near 0 or 1. You can only be easily swayed when you are ignorant. You can only remain easily swayed by remaining ignorant. The moment you acquire knowledge, instead of precisely measured ignorance, you are wearing lead-weighted boots.