“Will, probabilities are states of partial information, not objective properties of the problem. Unless you have reason to believe the data are wrong in a particular direction, your corrected estimate including meta-uncertainty is no different from the original as betting odds.”
This is not quite what I meant, I was more point out that you didn’t leave the us the opportunity to deny your data, which we would always have in the real world.
I’m also curious about the math for this. any math I have tried that assumes some error in the data seems to push the probabilities closer to 50% although by an unknown amount.
E.g. If you are trying to guess the bias in a (quantum) coin flip experiment and you have faulty detector of when things are heads and tails. Let us say you have 70 heads and 30 tails. If you assume false negatives (it is heads when you see tails) and false positives happen at the same rate and that they are independent. If the unknown error rate is E, you have 70E chance of at least one false positive and 30E chance of at least one false negative.
You might get the right results as you tend the amount of data to infinity but in limited trials things don’t look so rosy.
“Will, probabilities are states of partial information, not objective properties of the problem. Unless you have reason to believe the data are wrong in a particular direction, your corrected estimate including meta-uncertainty is no different from the original as betting odds.”
This is not quite what I meant, I was more point out that you didn’t leave the us the opportunity to deny your data, which we would always have in the real world.
I’m also curious about the math for this. any math I have tried that assumes some error in the data seems to push the probabilities closer to 50% although by an unknown amount.
E.g. If you are trying to guess the bias in a (quantum) coin flip experiment and you have faulty detector of when things are heads and tails. Let us say you have 70 heads and 30 tails. If you assume false negatives (it is heads when you see tails) and false positives happen at the same rate and that they are independent. If the unknown error rate is E, you have 70E chance of at least one false positive and 30E chance of at least one false negative.
You might get the right results as you tend the amount of data to infinity but in limited trials things don’t look so rosy.