I have some sympathy for the judge here, even as I wince. If in real life juries don’t understand Bayes and the actual effect of its use in court is to be grossly misused or make wild guesses sound formal and authoritative, then in the end you can’t have Bayes’s Theorem used formally in courts.
That isn’t what was going on in this case. The expert wasn’t presenting statistics to the jury (apparently that’s already forbidden).
The good news from this case (well, it’s news to me) is that the UK forensic science service both understands the statistics and has sensible written procedures for using them, which some of the examiners follow. But they then have to turn the likelihood ratio into a rather unhelpful form of words like ‘moderately strong scientific support’ (not to be confused with ‘moderate scientific support’, which is weaker), because bringing the likelihood ratios into court is forbidden.
(Bayes’ Theorem itself doesn’t really come into this case.)
Or to be a bit more precise: If you have good enough data to do anything useful with frequentist methods then you may use bayesian reasoning as well. What the judge forbade is using bayes to sound scientific when you can’t back up your priors.
What the judge forbade is using bayes to sound scientific when you can’t back up your priors.
The advantage of Bayesianism is that it is open about the relationship between prior beliefs, evidence, and updated beliefs.
Where there is enough data to use frequentist methods, that doesn’t imply one can produce relevant evidence for a case using those methods. I interpret you as agreeing with this based on your response, but feel free to clarify.
Jurors are not going to be able to tell to what extent frequentist methods produce valid evidence or not. It seems to me that if it is a good idea for judges to forbid using Bayesian reasoning because they can see where priors are arbitrary and are worried the jurors can’t, it is an even better idea for judges to forbid frequentist reasoning that doesn’t have a parallel permitted Bayesian process.
The two methods have similar relevance but differing opacity, and the clearer method is being punished because judges can understand its shortcomings. This leaves juries to deal with only evidence that the judge wasn’t able to understand.
Where there is enough data to use frequentist methods, that doesn’t imply one can produce relevant evidence for a case using those methods. I interpret you as agreeing with this based on your response, but feel free to clarify.
Jurors are not going to be able to tell to what extent frequentist methods produce valid evidence or not. It seems to me that if it is a good idea for judges to forbid using Bayesian reasoning because they can see where priors are arbitrary and are worried the jurors can’t, it is an even better idea for judges to forbid frequentist reasoning that doesn’t have a parallel permitted Bayesian process.
I agree with all of this. What I was trying to say is precisely that this isn’t about Bayes vs Fischer or whoever. Perhaps what I should have said to make that clearer is that the judge in this case did not (just) throw out Bayes, he threw out statistical inference.
I have some sympathy for the judge here, even as I wince. If in real life juries don’t understand Bayes and the actual effect of its use in court is to be grossly misused or make wild guesses sound formal and authoritative, then in the end you can’t have Bayes’s Theorem used formally in courts.
Is it bad that that sounds to me more like an argument against undereducated juries than against the use of Bayes in court?
That isn’t what was going on in this case. The expert wasn’t presenting statistics to the jury (apparently that’s already forbidden).
The good news from this case (well, it’s news to me) is that the UK forensic science service both understands the statistics and has sensible written procedures for using them, which some of the examiners follow. But they then have to turn the likelihood ratio into a rather unhelpful form of words like ‘moderately strong scientific support’ (not to be confused with ‘moderate scientific support’, which is weaker), because bringing the likelihood ratios into court is forbidden.
(Bayes’ Theorem itself doesn’t really come into this case.)
What are the options? Frequentist statistics, Bayesian statistics, both, or neither?
How many jurors understand statistical significance is surprise assuming one is wrong?
How many scientists understand the grant renewal case, or differences in differences?
No statistics at all.
Or to be a bit more precise: If you have good enough data to do anything useful with frequentist methods then you may use bayesian reasoning as well. What the judge forbade is using bayes to sound scientific when you can’t back up your priors.
Priors don’t come into it. The expert was presenting likelihood ratios directly (though in an obscure form of words).
+1 for biting the bullet. But...
The advantage of Bayesianism is that it is open about the relationship between prior beliefs, evidence, and updated beliefs.
Where there is enough data to use frequentist methods, that doesn’t imply one can produce relevant evidence for a case using those methods. I interpret you as agreeing with this based on your response, but feel free to clarify.
Jurors are not going to be able to tell to what extent frequentist methods produce valid evidence or not. It seems to me that if it is a good idea for judges to forbid using Bayesian reasoning because they can see where priors are arbitrary and are worried the jurors can’t, it is an even better idea for judges to forbid frequentist reasoning that doesn’t have a parallel permitted Bayesian process.
The two methods have similar relevance but differing opacity, and the clearer method is being punished because judges can understand its shortcomings. This leaves juries to deal with only evidence that the judge wasn’t able to understand.
I agree with all of this. What I was trying to say is precisely that this isn’t about Bayes vs Fischer or whoever. Perhaps what I should have said to make that clearer is that the judge in this case did not (just) throw out Bayes, he threw out statistical inference.