Inherited Improbabilities: Transferring the Burden of Proof

komponisto24 Nov 2010 3:40 UTC
46 points
58 commentsLW linkArchive

One person’s modus ponens is another’s modus tollens.

- Common saying among philosophers and other people who know what these terms mean.

If you believe A ⇒ B, then you have to ask yourself: which do I believe more? A, or not B?

- Hal Daume III, quoted by Vladimir Nesov.

Summary: Rules of logic have counterparts in probability theory. This post discusses the probabilistic analogue of modus tollens (the rule that if A=>B is true and B is false, then A is false), which is the inequality P(A) ≤ P(B)/​P(B|A). What this says, in ordinary language, is that if A strongly implies B, then proving A is approximately as difficult as proving B.

The appeal trial for Amanda Knox and Raffaele Sollecito starts today, and so to mark the occasion I thought I’d present an observation about probabilities that occurred to me while studying the “motivation document”(1), or judges’ report, from the first-level trial.

One of the “pillars” of the case against Knox and Sollecito is the idea that the apparent burglary in the house where the murder was committed—a house shared by four people, namely Meredith Kercher (the victim), Amanda Knox, and two Italian women—was staged. That is, the signs of a burglary were supposedly faked by Knox and Sollecito in order to deflect suspicion from themselves. (Unsuccessfully, of course...)

As the authors of the report, presiding judge Giancarlo Massei and his assistant Beatrice Cristiani, put it (p.44):

What has been explained up to this point leads one to conclude that the situation of disorder in Romanelli’s room and the breaking of the window constitute an artificially created production, with the purpose of directing investigators toward someone without a key to the entrance, who would have had to enter the house via the window whose glass had been broken and who would then have perpetrated the violence against Meredith that caused her death.

Now, even before examining “what has been explained up to this point”, i.e. the reasons that Massei and Cristiani (and the police before them) were led to this conclusion, we can pretty easily agree that if it is correct—that is, if Knox and Sollecito did in fact stage the burglary in Filomena Romanelli’s room—then it is extremely likely that they are guilty of participation in Kercher’s murder. After all, what are the chances that they just happened to engage in the bizarre offense of making it look like there was a burglary in the house, on the very same night as a murder occurred, in that very house? Now, one could still hypothetically argue about what their precise role was (e.g. whether they actually physically caused Kercher’s death, or merely participated in some sort of conspiracy to make the latter happen via the actions of known burglar and undisputed culprit Rudy Guede), and thus possibly about how severely they should be treated by the justice system; but in any case I think I’m on quite solid ground in asserting that a faked burglary by Knox and Sollecito would very strongly imply that Knox and Sollecito are criminally culpable in the death of Meredith Kercher.

...which is in fact quite a problem for Massei and Cristiani, as I’ll now explain.

Probability theory can and should be thought of as a quantitative version—indeed, a generalization—of the “rules of logic” that underpin Traditional Rationality. (Agreement with the previous sentence is essentially what it means to be a Bayesian.) One of these rules is this:

(1) If A implies B, then not-B implies not-A.

For example, all squares are in fact rectangles; which means that if something isn’t a rectangle, it can’t possibly be a square. Likewise, if “it’s raining” implies “the sidewalk is wet”, and you know the sidewalk isn’t wet, then you know it’s not raining.

The rule that gets you from “A implies B” and “A” to “B” is called modus ponens, which is Latin for “method that puts”. The rule that gets you from “A implies B” and “not-B” to “not-A” is called modus tollens, which is Latin for “method that takes away”. As the saying goes, and as we have just seen, they are really one and the same.

If, for a moment, we were to think about the Meredith Kercher case as a matter of pure logic—that is, where inferences were always absolutely certain, with zero uncertainty—then we could say that if we know that “burglary is fake” implies “Knox and Sollecito are guilty”, and we also know that the burglary was in fact fake, then we know that Knox and Sollecito are guilty.

But, of course, there’s another way to say the same thing: if we know that “burglary is fake” implies “Knox and Sollecito are guilty”, and we also know that Knox and Sollecito are innocent, then we know that the burglary wasn’t fake. (And that to the extent Massei and Cristiani say it was, they must be mistaken.)

In other words, so long as one accepts the implication “burglary fake ⇒ Knox and Sollecito guilty”, one can’t consistently hold that the burglary was fake and that Knox and Sollecito are innocent, but one can consistently hold either that the burglary was fake and Knox and Sollecito are guilty, or that Knox and Sollecito are innocent and the burglary was not fake.

The question of which of these two alternatives to believe thus reduces to the question of whether, given the evidence in the case, it’s more believable that Knox and Sollecito are guilty, or that the burglary was “authentic”. Massei and Cristiani, of course, aim to convince us that the latter is the more improbable.

But notice what this means! This means that the proposition that the burglary was fake assumes, or inherits, the same high burden of proof as the proposition that Knox and Sollecito committed murder! Unfortunately for Massei and Cristiani, there’s no way to “bootstrap up” from the mundane sort of evidence that seemingly suffices to show that a couple of youngsters engaged in some deception, to the much stronger sort of evidence required to prove that two honor students(2) with gentle personalities suddenly decided, on an unexpectedly free evening, to force a friend into a deadly sex game with a local drifter they barely knew, for the sake of a bit of thrill-seeking(3).

You may have noticed that, two paragraphs ago, I left the logical regime of implication, consistency, and absolute certainty, and entered the probability-theoretic realm of belief, uncertainty, and burdens of proof. So to make the point rigorous, we’ll have to switch from pure logic to its quantitative generalization, the mathematics of probability theory.
When logical statements are translated into their probabilistic analogues, a statement like “A is true” is converted to something like “P(A) is high”; “A implies B” becomes “A is (strong) evidence of B”; and rules such as (1) above turn into bounds on the probabilities of some hypotheses in terms of others.
Specifically, the translation of (1) into probabilistic language would be something like:
(2) If A is (sufficiently) strong evidence of B, and B is unlikely, then A is unlikely.
or
(2′) If A is (sufficiently) strong evidence of B, then the prior probability of A can’t be much higher than the prior probability of B.

Let’s prove this:

Suppose that A is strong evidence of B—that is, that P(B|A) is close to 1. We’ll represent this as P(B|A) ≥ 1-ε, where ε is a small number. Then, via Bayes’ theorem, this tells us that

or

so that

and thus

since P(A|B) ≤ 1. Hence we get an upper bound of P(B)/​(1-ε) on P(A). For instance, if P(B) is 0.001, and P(B|A) is at least 0.95, then P(A) can’t be any larger than 0.001/​0.95 = 0.001052...

Actually, there’s a simpler proof, direct from the definition of P(B|A), which goes like this: P(B|A) = P(A&B)/​P(A), whence P(A) = P(A&B)/​P(B|A) ≤ P(B)/​P(B|A). (Note the use of the conjunction rule: P(A&B) ≤ P(B).)

The statement

(3)

is a quantitative version of modus tollens, just as the equivalent statement

(4)

is a quantitative version of modus ponens. Assuming P(B|A) is high, what (4) says is that if P(A) is high, so is P(B); what (3) says is that if P(B) is low, so is P(A).

Or, in other words, that the improbability -- burden of proof—of B is transferred to, or inherited by, A.

...which means you cannot simultaneously believe that (1) Knox and Sollecito’s staging of the burglary would be strong evidence of their guilt; (2) proving their guilt is hard; and (3) proving they staged the burglary is easy. Something has to give; hard work must be done somewhere.

Of their 427-page report, Massei and Cristiani devote approximately 20 pages (mainly pp. 27-49) to their argument that the burglary was staged by Knox and Sollecito rather than being the work of known burglar Rudy Guede (including a strange section devoted to the refuting the hypothesis that the burglary was staged by Guede). But think about it: if they were really able to demonstrate this, they would scarcely have needed to bother writing the remaining 400-odd pages of the report! For, if it is granted that Knox and Sollecito staged the burglary, then, in the absence of any other explanation for the staging (like November 1 being Annual Stage-Burglary Day for some group to which Knox or Sollecito belonged) it easily follows with conviction-level confidence that they were involved in a conspiracy that resulted in the death of Meredith Kercher. You would hardly need to bother with DNA, luminol, or the cell phone traffic of the various “protagonists”.

Yet it doesn’t appear that Massei and Cristiani have much conception of the burden they face in trying to prove something that would so strongly imply their hugely a-priori-improbable ultimate thesis. Their arguments purporting to show that Knox and Sollecito faked the burglary are quite weak—and, indeed, are reminiscent of those used time and again by their lower-status counterparts, conspiracy theorists of all types, from 9/​11 “truthers” to the-Moon-landing-was-faked-ists. Here’s a sample, from p.39:

Additionally, the fragments of broken glass were scattered in a homogeneous manner on the internal and external windowsill, without any noticeable displacement and without any piece of glass being found on the surface below the window. This circumstance...rules out the possibility that the stone was thrown from outside the house to allow access inside via the window after the glass was broken. The climber, in leaning his hands and then his feet or knees on the windowsill, would have caused some of the glass to fall, or at least would have had to move some of the pieces lest they form a trap and cause injury. However, no piece of glass was found under the window and no sign of injury was discovered on the glass found in Romanelli’s room.

(The question to ask, when confronted with an argument like this, is: “rules out” with what confidence? If Massei and Cristiani think this is strong evidence against the hypothesis that the stone was thrown from outside the house, then that means they have a model that makes highly specific predictions about the behavior of glass fragments when a stone is thrown from inside, versus when it is thrown from outside. Predictions which can be tested(4). This is one reason why I advocate using numbers in arguments; if Massei and Cristiani had been required to think carefully enough to give a number, that would have forced them to examine their assumptions more critically, rather than stopping on plausible-sounding arguments consistent with their already-arrived-at bottom line.)
The impression one gets is that Massei and Cristiani thought, on some level, that all they needed to do was make the fake-burglary hypothesis sound coherent -- and that if they did so, that would count as a few points against Knox and Sollecito. They could then do the same thing with regard to the other pieces of evidence in the case, each time coming up with an explanation of the facts in terms of an assumption that Knox and Sollecito are guilty, and each time thereby scoring a few more points against them—points which would presumably add up to a substantial number by the end of the report.
But, of course, the mathematics of probability theory don’t work that way. It’s not enough for a hypothesis, such as that the apparent burglary in Filomena Romanelli’s room was staged, to merely be able to explain the data; it must do so better than its negation. And, in the absence of the assumption that Knox and Sollecito are guilty—if we’re presuming them to be innocent, as the law requires, or assigning a tiny prior probability to their guilt, as epistemic rationality requires—this contest is rigged. The standards for “explaining well” that the fake-burglary hypothesis has to meet in order to be taken seriously are much higher than those that its negation has to meet, because of the dependence relation that exists between the fake-burglary question and the murder question. Any hypothesis that requires the assumption that Knox and Sollecito are guilty of murder inherits the full “explanatory inefficiency penalty” (i.e. prior improbability) of the latter proposition.
If A implies B, then not-B implies not-A. It goes both ways.

Notes

(1) Some pro-guilt advocates have apparently produced a translation, but I haven’t looked at it and can’t vouch for it. Translations of passages appearing in this post are my own.

(2) One of whom, incidentally, is known to be enjoying Harry Potter and the Methods of Rationality -- so take that for whatever it’s worth.

(3) From p. 422 of the report:

The criminal acts turned out to be the result of purely accidental circumstances which came together to create a situation which, in the combination of the various factors, made the crimes against Meredith possible: Amanda and Raffaele, who happened to find themselves without any commitments, randomly met up with Rudy Guede (there is no trace of any planned appointment), and found themselves together in the house on Via Della Pergola where, that very evening, Meredith was alone. A crime which came into being, therefore, without any premeditation, without any animosity or rancorous feeling toward the victim...

(4) And sure enough, during the trial, the defense hired a ballistics expert who conducted experiments showing that a rock thrown from the outside would produce patterns of glass, etc. similar to what was found at the scene—results which forced the prosecutors to admit that the rock was probably thrown from the outside, but which were simply ignored by Massei and Cristiani! (See p. 229 of Sollecito’s appeal document, if you can read Italian.)

komponisto24 Nov 2010 3:40 UTC
46 points
58 commentsLW linkArchive