Followup to: The Bottom Line
We are warned that an argument made with a predecided conclusion does not evidentially entangle with the truth it claims to address, and thus is no evidence of that claim.
Recall that observation is evidence for hypothesis iff . What do the variables mean here?
: there’s a convincing argument for the predecided conclusion
: the predecided conclusion is really true
: probability that the clever arguer can make a clever argument, given that the conclusion is true
: likewise, given that the conclusion is false
What the commentary from “The Bottom Line” leaves out is that making a convincing argument is a nontrivial task. For many false claims, a clever arguer with ordinary resources cannot make a convincing argument. If it’s typically easy to make a convincing argument for something false, you’re convinced by the wrong things.
Thus , in this case, would usually be greater than . An argument for a claim from a clever arguer only clearly proves that the arguer wanted us to believe it. A convincing argument for a claim — and if the argument isn’t convincing, you’d ignore it — proves that the claim has convincing arguments for it accessible to that level of arguer, which is correlated with the claim being true.
But maybe I decided at the start that clever arguments for predecided conclusions are actual evidence, thereby breaking the entanglement of the rest of this essay. Well, are you convinced anyway?
Given that someone is being a clever arguer, the evidence of their argument may be taken to be evidence about the conclusion relative to what you might have expected they could come up with.
If a “clever arguer” says “Ten witnesses, named [...], all report seeing Bob murder Joe, and there are also multiple security cameras that caught it”, then that’s pretty damn strong evidence (assuming one follows up with the witnesses and gets the footage).
If a “clever arguer” says “Bob once got into a fight in middle school, so we see he has violent tendencies”, and that’s the best he managed to come up with, then it probably makes sense to update away from the conclusion that Bob murdered Joe.
Zvi has written about things of this ilk, vaguely connected to “bounded distrust”. I’ll see if I can find a link… Ok, this is a decent example of the general principle, although the counterparty isn’t a “clever arguer”:
The comment at the bottom there is important:
It’s a very lossy model to account for the arguer more than the arguments. It’s valid and important to remember that most fact aggregators are selective in what they say, with biases that don’t match the underlying truth (which they often don’t have access to either). But it’s not intended to be a formal literal “zero evidence”. It’s intended only as a reminder not to take the arguments at face value.
Note that in the example EY used, it IS literally zero evidence—the clever arguer doesn’t actually have any additional access to truth, or any evidence that the observer doesn’t already have. In this case, your inequality is false—there’s no difference between H and ~H in terms of probability of E.