There’s a big difference between summarizing data with a probability distribution, and practically flushing it away by thresholding it. The former is an associative, composable operation: You can keep doing it without really committing to any particular application of the data. The latter hands you a terminal object which is useless in the face of further information.
You don’t flush away your data, you are just running a request to it. You are not exterminating your own brain, you are only forming a categorical judgment. Like with tigers.
Sure. As I noted in the original post and also in response to your comments, sometimes it’s fine to do this. Sometimes it isn’t.
You are throwing away information when you threshold, period, end of that discussion. If that information was important, you made a mistake. It is easy to identify these situations because they have aberrant expected values in low-probability outcomes. And it is easy to avoid them, unlike with tigers.
You are throwing away information when you threshold, period, end of that discussion.
If the ‘information’ was noise, then it’s not information, it’s just data. Whether it counts as information depends upon relevance, which is partially what the threshold is for.
Neither, you just take a hit according to your scoring rule; but if you’re properly calibrated, it’ll be compensated for on average by 99 times that your 1-in-a-hundred chances don’t happen.
If I repeatedly claim I have a 1-in-2 chance of rolling snake eyes, you’d probably see me take repeated blows by a logarithmic scoring rule, which would suffice for bystanders to put less trust in my probability estimates. Eventually I should admit I’m poorly calibrated.
Of course if you’re talking about a probability 1 minus epsilon threshold, a miss on that claim is a huge penalty by a log scoring rule.
The probability value, which is what is being thrown away, is what is needed for a Bayesian analysis. Saying that that data is not useful means asserting that Bayesian analysis is not useful.
Precise Bayesian analysis is often impossible, and impossible can’t be useful. A notch down, take the difficulty of analysis as a counterbalance. Only if you can show that in a given situation you can do better by including that additional info, that is justified, not in general.
It throws away data. From a certain perspective, all intelligence does is throws away data that is not useful, to find information that is.
There’s a big difference between summarizing data with a probability distribution, and practically flushing it away by thresholding it. The former is an associative, composable operation: You can keep doing it without really committing to any particular application of the data. The latter hands you a terminal object which is useless in the face of further information.
You don’t flush away your data, you are just running a request to it. You are not exterminating your own brain, you are only forming a categorical judgment. Like with tigers.
Sure. As I noted in the original post and also in response to your comments, sometimes it’s fine to do this. Sometimes it isn’t.
You are throwing away information when you threshold, period, end of that discussion. If that information was important, you made a mistake. It is easy to identify these situations because they have aberrant expected values in low-probability outcomes. And it is easy to avoid them, unlike with tigers.
If the ‘information’ was noise, then it’s not information, it’s just data. Whether it counts as information depends upon relevance, which is partially what the threshold is for.
Link for more info
Of course, this is a jargon use in information theory. But that seems like the relevant domain.
It’s not useless, not by any means.
A threshold can be contradicted by a contrary-to-expectation observation. A statement of probability cannot, as long as it’s not an absolute.
If you believe that there’s only one-in-a-hundred chance of something happening, and it happens, were you right or wrong?
Neither, you just take a hit according to your scoring rule; but if you’re properly calibrated, it’ll be compensated for on average by 99 times that your 1-in-a-hundred chances don’t happen.
If I repeatedly claim I have a 1-in-2 chance of rolling snake eyes, you’d probably see me take repeated blows by a logarithmic scoring rule, which would suffice for bystanders to put less trust in my probability estimates. Eventually I should admit I’m poorly calibrated.
Of course if you’re talking about a probability 1 minus epsilon threshold, a miss on that claim is a huge penalty by a log scoring rule.
The probability value, which is what is being thrown away, is what is needed for a Bayesian analysis. Saying that that data is not useful means asserting that Bayesian analysis is not useful.
Precise Bayesian analysis is often impossible, and impossible can’t be useful. A notch down, take the difficulty of analysis as a counterbalance. Only if you can show that in a given situation you can do better by including that additional info, that is justified, not in general.
It often isn’t. Bayesianism isn’t the be-all, end-all of logic. It’s just another tool in the toolbox.