Good points, marred by what appears at first blush like double standards. Why are you willing to selectively discount some DNA evidence while you admit other ?
You say “the supposed Sollecito DNA on Meredith’s bra clasp just plain does not count”—what is that “supposed” doing in here ? The FOA site admits that the clasp was shown to carry small amounts of Sollecito’s DNA.
Why does it “not count” ? Admittedly, the handling of that evidence may not have been up to the standards normally demanded by the judicial system, but why should that matter to a Bayesian analysis ? All we’re interested in as Bayesians is the ratio between P(DNA on clasp|Sollecito guilty) and P(DNA on clasp|Sollecito not guilty).
The defense may well have its own convenient narratives about how Sollecito’s DNA “could have been transferred to the fastener in any number of ways” owing to carelessness on the part of the police. Those narratives are just as much noise as all the other noise you’ve pointed to. The details of US Department of Justice guidelines for forensics are also burdensome details for the defense.
The danger of coming across as arrogant—that is, more confident than your own entanglement with the evidence justifies, not the jurors’ or someone else’s—is that it provides others with an excuse to abandon the hard work of thinking rationally about the case, and letting themselves be swayed by the affective components: pretty pictures of Meredith or Amanda.
What I liked about your previous post was that, without any rah-rah propaganda about rationality, you outlined a procedure for laying out our current state of (un)certainty while minimizing some of the predictable biases. There was something delightful about the implausibly level-headed discussion thread it generated. We started with an open, almost playful invitation to rationality, in a spirit of inquiry.
Here we seem to be back to a spirit of established truth and advocacy, a spirit of “getting at the teacher’s answer”—we even get averaging of the students’ grades. I’m expecting the discussion to be more run-of-the mill, too. (Your Postscript is also mildly insulting; if someone is smart enough to arrive at the appropriate level of uncertainty in Knox’s guilt, then surely they are smart enough to figure out what to do about it.)
Here, I believe, is the bottom line. Make rationality fun, and people will play along without for a second realizing it’s hard work. Make rationality sound like something people ought to have, to be as smart and righteous as you—that’s a turn-off.
Can you explain your reasoning here, in terms of P(other folks’ DNA on clasp|Sollecito guilty) vs P(other folks’ DNA on clasp|Sollecito not guilty) ?
I can understand how this fact might be “suggestive” of something, but “suggestive” is the same kind of thinking as “suspicious”: it’s narrative rather than analytical.
It seems to me that the prosecution’s case against Sollecito relies quite heavily on the evidence they claim proves he was present at the crime scene since they have no other solid evidence against him.
The reasoning used by the prosecution is basically what Jaynes calls the ‘policeman’s syllogism’ in Probability Theory: The Logic of Science. The reasoning is of the form:
If A is true, then B becomes more plausible
B is true
Therefore, A becomes more plausible
Here A is (Sollecito was present at the crime scene) and B is (DNA tests on the bra clasp detected Sollecito’s DNA). If we use C to stand for our background knowledge then by Bayes theorem:
p(A|BC) = p(A|C) * (p(B|AC) / p(B|C))
The premise of the policeman’s syllogism “If A is true, then B becomes more plausible” takes the form
p(B|AC) > p(B|C)
And by Bayes theorem if this premise is true then:
p(A|BC) > p(A|C)
as stated in the syllogism. Now the significance of the evidence B depends on the magnitude of p(B|C) - the only way finding B to be true can greatly increase the plausibility of A is if p(B|C) is very small relative to p(B|AC). In other words, the prosecution’s argument rests on the background probability of finding Sollectio’s DNA on the bra clasp being very low relative to the probability of finding it if he were present at the crime scene.
Now it seems to me that the fact that the DNA of several other unidentified individuals (who it is not suggested were present at the crime scene) was also found on the bra clasp indicates that p(B|C) is not so much smaller than p(B|AC). B is only strong evidence for A if DNA on the clasp is much more likely if the person was present at the crime scene but we have several counter examples of DNA on the clasp from individuals who were not at the crime scene so we have reason to doubt that p(B|AC) is much greater than p(B|C) and therefore reason to doubt the significance of the evidence.
And by and large I agree with the analysis—that is, I agree that how much weight to give to that particular evidence is determined by your estimates of P(B|AC) and p(B|C).
We may yet disagree on these, but if we do it should be on the basis of models that further evidence can in principle confirm or rule out, for instance whose DNA exactly was found on the clasp—does it match the investigators’ ? They were at the crime scene. Contamination of that sort would help (in a Bayesian sense) the prosecution, not the defense.
What I take issue with is to say that something “does not count” when we have a previous commitment to take into account every bit of evidence available to us. Either we use Bayesian standards of inquiry, or judicial standards of inquiry, but we do not cherry-pick which is convenient to a given point we want to make.
Check out the blog ScienceSpheres by Mark Waterbury. He discusses at length the issues of negative controls, field controls, and pinpoints the problems with the LCN DNA analysis. One of his key points is that the mistakes in the evidence gathering and testing aren’t hit or miss—they are consistent—which reveals a pattern of intention.
Why are you willing to selectively discount some DNA evidence while you admit other ?
Because the 2 key pieces of evidence should be discounted because they were not arrived at by using the same type of test and were collected differently. All the other DNA testing was done using standard DNA testing in a lab that was nominally set up to do it but which did not always follow all guidelines and procedures and who did not release all data for defense experts to evaluate.
NO DNA that helped the prosecutions case was found initially on the 2 items… the bra clasp and the knife. Regular testing on the rest of the bra had strong findings for Meredith and Rudy. Amanda’s DNA was on the knife handle and the bra clasp did have Meredith’s as well.
Now the problem comes… the DNA specialist then attempted doing what amounts to Low Count Number (LCN) DNA test which is still experimental in most places and while now accepted in the UK it requires a very specialized expensive lab that must meet many stringent minimum requirements. The tester in this situation had never done this before, the lab was fundamentally lacking in every way to do this test and come up with verifiable, trustworthy data. And yet they did allege that they found MKs DNA or something like it… on the knife… not blood.… no blood on the knife. The DNA would have been in picogram amounts originally and easily contaminated in that lab or elsewhere by the smallest flecks of skin or dust. And there are the same problems with the bra clasp except that along with RS’s alleged DNA there are 5 other peoples… unidentified. There can be little doubt that all the DNA and “all” is a very very tiny amount, much smaller than regular standard DNA can reliable even detect as being there...
Good points, marred by what appears at first blush like double standards. Why are you willing to selectively discount some DNA evidence while you admit other ?
You say “the supposed Sollecito DNA on Meredith’s bra clasp just plain does not count”—what is that “supposed” doing in here ? The FOA site admits that the clasp was shown to carry small amounts of Sollecito’s DNA.
Why does it “not count” ? Admittedly, the handling of that evidence may not have been up to the standards normally demanded by the judicial system, but why should that matter to a Bayesian analysis ? All we’re interested in as Bayesians is the ratio between P(DNA on clasp|Sollecito guilty) and P(DNA on clasp|Sollecito not guilty).
The defense may well have its own convenient narratives about how Sollecito’s DNA “could have been transferred to the fastener in any number of ways” owing to carelessness on the part of the police. Those narratives are just as much noise as all the other noise you’ve pointed to. The details of US Department of Justice guidelines for forensics are also burdensome details for the defense.
The danger of coming across as arrogant—that is, more confident than your own entanglement with the evidence justifies, not the jurors’ or someone else’s—is that it provides others with an excuse to abandon the hard work of thinking rationally about the case, and letting themselves be swayed by the affective components: pretty pictures of Meredith or Amanda.
What I liked about your previous post was that, without any rah-rah propaganda about rationality, you outlined a procedure for laying out our current state of (un)certainty while minimizing some of the predictable biases. There was something delightful about the implausibly level-headed discussion thread it generated. We started with an open, almost playful invitation to rationality, in a spirit of inquiry.
Here we seem to be back to a spirit of established truth and advocacy, a spirit of “getting at the teacher’s answer”—we even get averaging of the students’ grades. I’m expecting the discussion to be more run-of-the mill, too. (Your Postscript is also mildly insulting; if someone is smart enough to arrive at the appropriate level of uncertainty in Knox’s guilt, then surely they are smart enough to figure out what to do about it.)
Here, I believe, is the bottom line. Make rationality fun, and people will play along without for a second realizing it’s hard work. Make rationality sound like something people ought to have, to be as smart and righteous as you—that’s a turn-off.
The Friends of Amanda site claims:
If that is true then it suggests to me that finding Sollecito’s DNA as well is not very strong evidence for anything.
Can you explain your reasoning here, in terms of P(other folks’ DNA on clasp|Sollecito guilty) vs P(other folks’ DNA on clasp|Sollecito not guilty) ?
I can understand how this fact might be “suggestive” of something, but “suggestive” is the same kind of thinking as “suspicious”: it’s narrative rather than analytical.
It seems to me that the prosecution’s case against Sollecito relies quite heavily on the evidence they claim proves he was present at the crime scene since they have no other solid evidence against him.
The reasoning used by the prosecution is basically what Jaynes calls the ‘policeman’s syllogism’ in Probability Theory: The Logic of Science. The reasoning is of the form:
If A is true, then B becomes more plausible
B is true
Therefore, A becomes more plausible
Here A is (Sollecito was present at the crime scene) and B is (DNA tests on the bra clasp detected Sollecito’s DNA). If we use C to stand for our background knowledge then by Bayes theorem:
The premise of the policeman’s syllogism “If A is true, then B becomes more plausible” takes the form
And by Bayes theorem if this premise is true then:
as stated in the syllogism. Now the significance of the evidence B depends on the magnitude of p(B|C) - the only way finding B to be true can greatly increase the plausibility of A is if p(B|C) is very small relative to p(B|AC). In other words, the prosecution’s argument rests on the background probability of finding Sollectio’s DNA on the bra clasp being very low relative to the probability of finding it if he were present at the crime scene.
Now it seems to me that the fact that the DNA of several other unidentified individuals (who it is not suggested were present at the crime scene) was also found on the bra clasp indicates that p(B|C) is not so much smaller than p(B|AC). B is only strong evidence for A if DNA on the clasp is much more likely if the person was present at the crime scene but we have several counter examples of DNA on the clasp from individuals who were not at the crime scene so we have reason to doubt that p(B|AC) is much greater than p(B|C) and therefore reason to doubt the significance of the evidence.
Now that is analytical.
And by and large I agree with the analysis—that is, I agree that how much weight to give to that particular evidence is determined by your estimates of P(B|AC) and p(B|C).
We may yet disagree on these, but if we do it should be on the basis of models that further evidence can in principle confirm or rule out, for instance whose DNA exactly was found on the clasp—does it match the investigators’ ? They were at the crime scene. Contamination of that sort would help (in a Bayesian sense) the prosecution, not the defense.
What I take issue with is to say that something “does not count” when we have a previous commitment to take into account every bit of evidence available to us. Either we use Bayesian standards of inquiry, or judicial standards of inquiry, but we do not cherry-pick which is convenient to a given point we want to make.
Very well said.
Check out the blog ScienceSpheres by Mark Waterbury. He discusses at length the issues of negative controls, field controls, and pinpoints the problems with the LCN DNA analysis. One of his key points is that the mistakes in the evidence gathering and testing aren’t hit or miss—they are consistent—which reveals a pattern of intention.
Why are you willing to selectively discount some DNA evidence while you admit other ?
Because the 2 key pieces of evidence should be discounted because they were not arrived at by using the same type of test and were collected differently. All the other DNA testing was done using standard DNA testing in a lab that was nominally set up to do it but which did not always follow all guidelines and procedures and who did not release all data for defense experts to evaluate.
NO DNA that helped the prosecutions case was found initially on the 2 items… the bra clasp and the knife. Regular testing on the rest of the bra had strong findings for Meredith and Rudy. Amanda’s DNA was on the knife handle and the bra clasp did have Meredith’s as well.
Now the problem comes… the DNA specialist then attempted doing what amounts to Low Count Number (LCN) DNA test which is still experimental in most places and while now accepted in the UK it requires a very specialized expensive lab that must meet many stringent minimum requirements. The tester in this situation had never done this before, the lab was fundamentally lacking in every way to do this test and come up with verifiable, trustworthy data. And yet they did allege that they found MKs DNA or something like it… on the knife… not blood.… no blood on the knife. The DNA would have been in picogram amounts originally and easily contaminated in that lab or elsewhere by the smallest flecks of skin or dust. And there are the same problems with the bra clasp except that along with RS’s alleged DNA there are 5 other peoples… unidentified. There can be little doubt that all the DNA and “all” is a very very tiny amount, much smaller than regular standard DNA can reliable even detect as being there...
Add to that no control tests were done and the “evidence” involving these 2 items evaporates. They are rightly to be judged differently from all the other DNA evidence. This looks like a DNA fishing expedition done after no DNA evidence for RS & AK turned up in MK’S bedroom. check these for details http://www.sciencespheres.com/2009/10/methods-of-polizia-pseudoscientificaa.html http://freeaman.001webs.com/pdfs/LCN_DNA_I.pdf http://freeaman.001webs.com/pdfs/LCN_DNA_II.pdf
the LCN dna test was only done on the knife; the bra clasp was a regular DNA test, i believe.