Of course you can’t derive it—you gave a counterexample!
It’s not a real counterexample. The data sets aren’t actually independent. The real trick is that I try to convince you that the 2 datasets being used are independent because they don’t intersect. Common definitions of independence would say they are independent because you can’t compute any correlation between them. But they both have the same underlying probability distribution generated from the same source. I’m confused about what “independence” should mean in this case.
it seems to be written in a standard formal language and you have to interpret it with A and B as random variables, “independent” meaning independence of random variables. Then the formal statement is false by the example of three independent coin flips. When I first posted my comment, I had a panicked moment and deleted it until I realized that the three coins example is a counter example; whether it is the same as your example is not important.
Yes, the two data sets are not independent, if you’re not sure how coins are weighted. Two flips of a coin weighted in an unknown way are not independent random variables. But that’s good, because if they were independent in that way, their evidence wouldn’t add. But since they are independent conditional on the thing we want to measure, their evidence does add.
I repeated your original formulation of the consequence of independent evidence, but it is not correct. If you have two independent pieces of evidence that would each send you to 90% certainty, you do not conclude 99% certainty. It depends on your prior! If your prior were 50%, you conclude 98%. If your prior were 90%, neither piece of evidence told you anything, so together you learn nothing! (Moreover, if A and B are empty observations, that is a counterexample to pretty much any formulation.)
When is evidence independent? Evidence is the log of the likelihood ratio, which is what shows up in a bayesian update. The likelihood ratio for X involves only probabilities conditional on X. Thus independence of A and B conditional on X is exactly what we need for the probabilities to multiply; for the evidence of A and B to be the sum of the evidence of the individual events. In particular, the evidence from two flips of a single weighted coin give independent evidence about the weight.
It’s not a real counterexample. The data sets aren’t actually independent. The real trick is that I try to convince you that the 2 datasets being used are independent because they don’t intersect. Common definitions of independence would say they are independent because you can’t compute any correlation between them. But they both have the same underlying probability distribution generated from the same source. I’m confused about what “independence” should mean in this case.
When you make the statement
it seems to be written in a standard formal language and you have to interpret it with A and B as random variables, “independent” meaning independence of random variables. Then the formal statement is false by the example of three independent coin flips. When I first posted my comment, I had a panicked moment and deleted it until I realized that the three coins example is a counter example; whether it is the same as your example is not important.
Yes, the two data sets are not independent, if you’re not sure how coins are weighted. Two flips of a coin weighted in an unknown way are not independent random variables. But that’s good, because if they were independent in that way, their evidence wouldn’t add. But since they are independent conditional on the thing we want to measure, their evidence does add.
I repeated your original formulation of the consequence of independent evidence, but it is not correct. If you have two independent pieces of evidence that would each send you to 90% certainty, you do not conclude 99% certainty. It depends on your prior! If your prior were 50%, you conclude 98%. If your prior were 90%, neither piece of evidence told you anything, so together you learn nothing! (Moreover, if A and B are empty observations, that is a counterexample to pretty much any formulation.)
When is evidence independent?
Evidence is the log of the likelihood ratio, which is what shows up in a bayesian update. The likelihood ratio for X involves only probabilities conditional on X. Thus independence of A and B conditional on X is exactly what we need for the probabilities to multiply; for the evidence of A and B to be the sum of the evidence of the individual events. In particular, the evidence from two flips of a single weighted coin give independent evidence about the weight.