Essentially all scientific fields rely heavily on statistics.
What does this mean? If they need statistics to reach the truth, then undermining the statistics is a very big deal.
I can see only a few possibilities:
They really need statistics, but they are making random errors in their statistics and getting random results. The field is worthless.
They are reaching the correct conclusions through non-statistical scientific methods and the statistics is window-dressing. (or perhaps they real method is statistical, but much simpler and more robust than they claim)
They are reaching wrong conclusions through unspecified wrong methods and the statistics is window dressing. (How can you distinguish this from the previous?)
The leaders of the field do correct statistics and reach the correct conclusions. Everyone else copies their conclusions and messes up the statistics. It should be easy to find the leaders and check their statistics.
How about “their basic reasoning stands, but they are not being very rigorous with their statistics, so there may be some small errors in p-values and some interpretations”.
A bit like if I said “I see a lot of light coming through the window, and it’s 4 PM, so it’s probably sunny outside”, and tried to formalize it statistically. There may be plenty of mistakes in the formalization, but it probably still is sunny.
Doesn’t this amount to a rejection of Chris’s “Essentially all scientific fields rely heavily on statistics”? Am I using “rely” differently than everyone else?
How does that differ from my point 2, especially “more robust than they claim”?
Five: They have mostly correct beliefs because they mostly do statistics correctly. This leads them to only test things with high prior probability, where they can screw up the statistics and still get the right answers. However, if they never did the statistics properly, they would drift away from these correct beliefs.
What does this mean?
If they need statistics to reach the truth, then undermining the statistics is a very big deal.
I can see only a few possibilities:
They really need statistics, but they are making random errors in their statistics and getting random results. The field is worthless.
They are reaching the correct conclusions through non-statistical scientific methods and the statistics is window-dressing. (or perhaps they real method is statistical, but much simpler and more robust than they claim)
They are reaching wrong conclusions through unspecified wrong methods and the statistics is window dressing. (How can you distinguish this from the previous?)
The leaders of the field do correct statistics and reach the correct conclusions. Everyone else copies their conclusions and messes up the statistics. It should be easy to find the leaders and check their statistics.
How about “their basic reasoning stands, but they are not being very rigorous with their statistics, so there may be some small errors in p-values and some interpretations”.
A bit like if I said “I see a lot of light coming through the window, and it’s 4 PM, so it’s probably sunny outside”, and tried to formalize it statistically. There may be plenty of mistakes in the formalization, but it probably still is sunny.
Doesn’t this amount to a rejection of Chris’s “Essentially all scientific fields rely heavily on statistics”?
Am I using “rely” differently than everyone else?
How does that differ from my point 2, especially “more robust than they claim”?
Five: They have mostly correct beliefs because they mostly do statistics correctly. This leads them to only test things with high prior probability, where they can screw up the statistics and still get the right answers. However, if they never did the statistics properly, they would drift away from these correct beliefs.