You need a probability distribution over something like pairs (crash probability without this line, crash probability with this line). So the likelihood for (p,q) is Pr(2 crashes of 7 with the line | p) Pr(0 crashes of 10 without the line | q) = 21 p^2 (1-p)^5 (1-q)^10, and presumably you’ve got a prior that prefers small p, small q, and p ~= q.
Then you could look deeper into the kind of thing that might be going on and consider hypotheses like “there’s a race condition that happens early in the program when debug logging is active, because of the extra time taken writing to the log” but it’s going to be really tough assigning the relevant conditional probabilities. In practice, if a lot of your probability in the initial (p,q) thing ends up in regions where p<q by a substantial margin, it’s probably reasonable to say that adding the line causes the program to misbehave.
So, if I understand correctly, my posterior probability is a distribution over the (p, q) space; and presumably, the particular data I’ve got becomes more likely as p approaches 2⁄7 and q approaches 0. So if I had, say, a uniform prior, or a prior that was uniform in the lower-left corner where both p and q are small, then I’d move towards zero q and nonzero p, or in other words “The line causes the crash”. That seems to make sense.
You need a probability distribution over something like pairs (crash probability without this line, crash probability with this line). So the likelihood for (p,q) is Pr(2 crashes of 7 with the line | p) Pr(0 crashes of 10 without the line | q) = 21 p^2 (1-p)^5 (1-q)^10, and presumably you’ve got a prior that prefers small p, small q, and p ~= q.
Then you could look deeper into the kind of thing that might be going on and consider hypotheses like “there’s a race condition that happens early in the program when debug logging is active, because of the extra time taken writing to the log” but it’s going to be really tough assigning the relevant conditional probabilities. In practice, if a lot of your probability in the initial (p,q) thing ends up in regions where p<q by a substantial margin, it’s probably reasonable to say that adding the line causes the program to misbehave.
So, if I understand correctly, my posterior probability is a distribution over the (p, q) space; and presumably, the particular data I’ve got becomes more likely as p approaches 2⁄7 and q approaches 0. So if I had, say, a uniform prior, or a prior that was uniform in the lower-left corner where both p and q are small, then I’d move towards zero q and nonzero p, or in other words “The line causes the crash”. That seems to make sense.