A Problem with Human Intuition about Conventional Statistics:
As an aspiring scientist, I hold the Truth above all. As Hodgell once said, “That which can be destroyed by the truth should be.” But what if the thing that is holding our pursuit of the Truth back is our own system? I will share an example of an argument I overheard between a theist and an atheist once—showing an instance where human intuition might fail us.
*General Transcript*
Atheist: Prove to me that God exists!
Theist: He obviously exists – can’t you see that plants growing, humans thinking, [insert laundry list here], is all His work?
Atheist: Those can easily be explained by evolutionary mechanisms!
Theist: Well prove to me that God doesn’t exist!
Atheist: I don’t have to! There may be an invisible pink unicorn baby flying around my head, there is probably not. I can’t prove that there is no unicorn, that doesn’t mean it exists!
Theist: That’s just complete reductio ad ridiculo, you could do infrared, polaroid, uv, vacuum scans, and if nothing appears it is statistically unlikely that the unicorn exists! But God is something metaphysical, you can’t do that with Him!
Atheist: Well Nietzsche killed metaphysics when he killed God. God is dead!
Theist: That is just words without argument. Can you actually…..
As one can see, the biggest problem is determining burden of proof. Statistically speaking, this is much like the problem of defining the null hypothesis.
A theist would define: H0 : God exists. Ha: God does not exist.
An atheist would define: H0: God does not exist. Ha God does exist.
Both conclude that there is no significant evidence hinting at Ha over H 0. Furthermore, and this is key, they both accept the null hypothesis. The correct statistical term for the proper conclusion if insignificant evidence exists for the acceptance of the alternate hypothesis is that one fails to reject the null hypothesis. However, human intuition fails to grasp this concept, and think in black and white, and instead we tend to accept the null hypothesis.
This is not so much a problem with statistics as it is with human intuition. Statistics usually take this form because simultaneous 100+ hypothesis considerations are taxing on the human brain. Therefore, we think of hypotheses to be defended or attacked, but not considered neutrally.
Considered a Bayesian outlook on this problem.
There are two possible outcomes: At least one deity exists(D). No deities exist(N).
Let us consider the natural evidence (Let’s call this E) before us.
P(D+N) = 1. P[(D+N)|E] = 1. P(D|E) + P(N|E) = 1. P(D|E) = 1- P(N|E).
Although the calculation of the prior probability of the probability of god existing is rather strange, and seems to reek of bias, I still argue that this is a better system of analysis than just the classical H0 and Ha, because it effectively compares the probability of D and N with no bias inherent in the brain’s perception of the system.
Example such as these, I believe, show the flaws that result from faulty interpretations of the classical system. If instead we introduced a Bayesian perspective – the faulty interpretation would vanish.
This is a case for the expanded introduction of Bayesian probability theory. Even if cannot be applied correctly to every problem, even if it is apparently more complicated than the standard method they teach in statistics class ( I disagree here), it teaches people to analyze situations from a more objective perspective.
And if we can avoid Truth-seekers going awry due to simple biases such as those mentioned above, won’t we be that much closer to finding Truth?
I’ve tried to explain this when arguing with theists, and it sometimes creates the following unintentional side effect:
The problem, of course, is that the theist forgot (or doesn’t understand) the distinction between “(the Judeo-Christian) God exists” and “at least one deity exists.” It’s really important to stress that the search space of possible gods is huge, otherwise you will create even more confusion.
Overall, though, I definitely agree with the main point of this post. Upvoted.
Quite honestly, I think a bigger problem is theists assuming that P(E|D) = 100%. That given a deity or more exists, they automatically assume the world would turn out like this—I would actually argue the opposite, that the number is very low.
Even assuming an omniscient, omnipotent, omnibenevolent God, he could have still, I argue at least, have made the choice to remove our free will “for our own good”. Even if P(E|D) is high, in no way is it close to 100%.
Furthermore, you can never assume a 100% probability!!! (http://yudkowsky.net/rational/technical). You could go to rationalist hell for that!
Conditional probabilities are allowed to be 100%, because they are probability ratios. In particular, P(A|A) is 100% by definition.
But P(E|D) is not 100% by any definition. Conditional probabilities are only 100% if
D-->E. And if that was true, why does this argument exist?
The reason they are assuming P(E|D) = 100% is probably because they are only envisioning what one particular god would do, not the whole search space: they are asking “What would God do?” instead of “What would X percentage of the zillion possible gods in the search space do?” The hard part is getting them to realize that P(E|D) includes Zeus, Loki, and the FSM as well as Jehovah.
That will change!
More seriously though...
Well, not really. The null and alternative hypotheses in frequentist statistics are defined in terms of their model complexity, not our prior beliefs (that would be Bayesian!). Specifically, the null hypothesis represents the model with fewer free parameters.
You might still face some sort of statistical disagreement with the theist, but it would have to be a disagreement over which hypothesis is more/less parsimonious—which is really a rather different argument than what you’ve outlined (and IMO, one that the theist would have a hard time defending).
I’m not saying that the frequentist statistical belief logic actually goes like that above. What I’m saying is that is how many people tend to wrongly interpret such statistics to define their own null hypothesis in the way I outlined in the post.
As I’ve said before, the MOST common problem is not the actual statistics, but how the ignorant interpret that statistics. I am merely saying, I would prefer Bayesian statistics to be taught because it is much harder to botch up and read our own interpretation into it. (For one, it is ruled by a relatively easy formula)
Also, isn’t model complexity quite hard to determine with the statements “God exists” and “God does not exist”. Isn’t the complexity in this sense subject to easy bias?
But that’s not right. The problem that your burden of proof example describes is a problem of priors. The theist and the atheist are starting with priors that favor different hypotheses. But priors (notoriously!) don’t enter into the NHST calculus. Given two statistical models, one of which is a nested subset of the other (this is required in order to directly compare them), there is not a choice of which is the null: the null model is the one with fewer parameters (i.e., it is the nested subset). It isn’t up for debate.
There are other problems with NHST—as you point out later in the post, some people have a hard time keeping straight just what the numbers are telling them—but the issue I highlighted above isn’t one of them for me.
Yes. As you noted in your OP, forcing this pair of hypotheses into a strictly statistical framework is awkward no matter how you slice it. Statistical hypotheses ought to be simple empirical statements.
It seems like there should be a paragraph after the Bayesian conservation of probabilities bit. Some kind of explanation. The way I’m interpreting your point, after that line, is that with the null hypothesis people, they are arguing over what is the default based on insufficient evidence, whereas they should be recognizing that any evidence for one is evidence against the other position. This seems to play into your italics point, that the hypotheses are battling, not neutral. Are you trying to say that we should treat conflicting hypotheses as dueling komodo dragons, or that Bayesian analysis is better at considering things neutrally, and therefore a better way to reach consensus? Or something different entirely?