.I hear you, but R has enough fully-automated testing tools that it’s much simpler for me to just run the appropriate test and see what pops out the other end. (Also, THANK YOU for mentioning Chebyshev, I can’t believe I’d never heard of that inequality before and it’s EXACTLY my kind of thing)
.I think (?) you’re operating on the wrong level of meta here. A t-test uses both the mean and the variance of the distribution(s) you feed it, and that’s true whether or not it’s being used to test a correlation. The CLT will not save us, because the single (admittedly gaussian-distributed) datapoint representing the mean has a variance of zero. (Something I could have done—in fact, something I remember doing much earlier in my career, back when I was better at identifying problems than finding expedient solutions—was to group not-necessarily-normal datapoints together into batches of about twenty, take the averages per-batch, and then t-test the lists of those: it was a ridiculous waste of statistical power, but it was valid!)
.That’s an excellent idea. My excuse for not doing that is that I was prioritising pointedly-not-getting-things-wrong over actually-getting-things-right; my reason is that I just didn’t think of it and I’m too lazy (and data-purist) to go back and try that now.
Responses to your differences:
.I hear you, but R has enough fully-automated testing tools that it’s much simpler for me to just run the appropriate test and see what pops out the other end. (Also, THANK YOU for mentioning Chebyshev, I can’t believe I’d never heard of that inequality before and it’s EXACTLY my kind of thing)
.I think (?) you’re operating on the wrong level of meta here. A t-test uses both the mean and the variance of the distribution(s) you feed it, and that’s true whether or not it’s being used to test a correlation. The CLT will not save us, because the single (admittedly gaussian-distributed) datapoint representing the mean has a variance of zero. (Something I could have done—in fact, something I remember doing much earlier in my career, back when I was better at identifying problems than finding expedient solutions—was to group not-necessarily-normal datapoints together into batches of about twenty, take the averages per-batch, and then t-test the lists of those: it was a ridiculous waste of statistical power, but it was valid!)
.That’s an excellent idea. My excuse for not doing that is that I was prioritising pointedly-not-getting-things-wrong over actually-getting-things-right; my reason is that I just didn’t think of it and I’m too lazy (and data-purist) to go back and try that now.
The dataset is, at time of writing, still up at https://gist.github.com/ncase/74ae97cb74893a0c540274b44f550503. I’d love to see what you throw at it.