Can you point to examples of these “holy wars”? I haven’t encountered something I’d describe like that, so I don’t know if we’ve been seeing different things, or just interpreting it differently.
To me it looks like a tension between a method that’s theoretically better but not well-established, and a method that is not ideal but more widely understood so more convenient—a bit like the tension between the metric and imperial systems, or between flash and html5.
The term “holy war” or “religious war” is often used to describe debates where people advocate for a side with an intensity disproportionate to the stakes, (e.g. the proper pronunciation of “gif”, vi vs. emacs, surrogate vs. natural primary keys in the RDBM). That’s how I read the OP, and it’s fitting in context.
Dude, I’m being genuinely curious about what “holy wars” he’s talking about. So far I got:
a definition of “holy war” in this context
a snotty “shut up, only statisticians are allowed to talk about this topic”
… but zero actual answers, so I can’t even tell if he’s talking about some stupid overblown bullshit, or if he’s just exaggerating what is actually a pretty low-key difference in opinion.
A “holy war” between Bayesians and frequentists exists in the modern academic literature for statistics, machine learning, econometrics, and philosophy (this is a non-exhaustive list).
Bradley Efron, who is arguably the most accomplished statistician alive, wrote the following in a commentary for Science in 2013 [1]:
The term “controversial theorem” sounds like an oxymoron, but Bayes’ theorem has played this part for two-and-a-half centuries. Twice it has soared to scientific celebrity, twice it has crashed, and it is currently enjoying another boom. The theorem itself is a landmark of logical reasoning and the first serious triumph of statistical inference, yet is still treated with suspicion by most statisticians. There are reasons to believe in the staying power of its current popularity, but also some signs of trouble ahead.
[...]
Bayes’ 1763 paper was an impeccable exercise in probability theory. The trouble and the subsequent busts came from overenthusiastic application of the theorem in the absence of genuine prior information, with Pierre-Simon Laplace as a prime violator. Suppose that in the twins example we lacked the prior knowledge that one-third of twins are identical. Laplace would have assumed a uniform distribution between zero and one for the unknown prior probability of identical twins, yielding 2⁄3 rather than 1⁄2 as the answer to the physicists’ question. In modern parlance, Laplace would be trying to assign an “uninformative prior” or “objective prior”, one having only neutral effects on the output of Bayes’ rule. Whether or not this can be done legitimately has fueled the 250-year controversy.
Frequentism, the dominant statistical paradigm over the past hundred years, rejects the use of uninformative priors, and in fact does away with prior distributions entirely. In place of past experience, frequentism considers future behavior. An optimal estimator is one that performs best in hypothetical repetitions of the current experiment. The resulting gain in scientific objectivity has carried the day, though at a price in the coherent integration of evidence from different sources, as in the FiveThirtyEight example.
The Bayesian-frequentist argument, unlike most philosophical disputes, has immediate practical consequences.
In another paper published in 2013, Efron wrote [2]:
The two-party system [Bayesian and frequentist] can be upsetting to statistical consumers, but it has been a good thing for statistical researchers — doubling employment, and spurring innovation within and between the parties. These days there is less distance between Bayesians and frequentists, especially with the rise of objective Bayesianism, and we may even be heading toward a coalition government.
The two philosophies, Bayesian and frequentist, are more orthogonal than antithetical. And of course, practicing statisticians are free to use whichever methods seem better for the problem at hand — which is just what I do.
Thirty years ago, Efron was more critical of Bayesian statistics [3]:
A summary of the major reasons why Fisherian and NPW [Neyman–Pearson–Wald] ideas have shouldered Bayesian theory aside in statistical practice is as follows:
Ease of use: Fisher’s theory in particular is well set up to yield answers on an easy and almost automatic basis.
Model building: Both Fisherian and NPW theory pay more attention to the preinferential aspects of statistics.
Division of labor: The NPW school in particular allows interesting parts of a complicated problem to be broken off and solved separately. These partial solutions often make use of aspects of the situation, for example, the sampling plan, which do not seem to help the Bayesian.
Objectivity: The high ground of scientific objectivity has been seized by the frequentists.
None of these points is insurmountable, and in fact, there have been some Bayesian efforts on all four. In my opinion a lot more such effort will be needed to fulfill Lindley’s prediction of a Bayesian 21st century.
The following bit of friendly banter in 1965 between M. S. Bartlett and John W. Pratt shows that the holy war was ongoing 50 years ago [4]:
Bartlett: I am not being altogether facetious in suggesting that, while non-Bayesians should make it clear in their writings whether they are non-Bayesian Orthodox or non-Bayesian Fisherian, Bayesians should also take care to distinguish their various denominations of Bayesian Epistemologists, Bayesian Orthodox and Bayesian Savages. (In fairness to Dr Good, I could alternatively have referred to Bayesian Goods; but, oddly enough, this did not sound so good.)
Pratt: Professor Bartlett is correct in classifying me a Bayesian Savage, though I might take exception to his word order. On the whole, I would rather be called a Savage Bayesian than a Bayesian Savage. Of course I can quite see that Professor Bartlett might not want to admit the possibility of a Good Bayesian.
Dude, I’m being genuinely curious about what “holy wars” he’s talking about.
For lots of “holy war” anecdotes, see The Theory That Would Not Die by Sharon Bertsch McGrayne.
...I can’t even tell if he’s talking about some stupid overblown bullshit, or if he’s just exaggerating what is actually a pretty low-key difference in opinion.
Do you consider personal insults, accusations of fraud, or splitting academic departments along party lines to be “a pretty low-key difference in opinion”? If so, then it is “overblown bullshit,” otherwise it isn’t.
Can you point to examples of these “holy wars”? I haven’t encountered something I’d describe like that, so I don’t know if we’ve been seeing different things, or just interpreting it differently.
Various bits of Jaynes’s “Confidence intervals vs Bayesian intervals” seem holy war-ish to me. Perhaps the juiciest bit (from pages 197-198, or pages 23-24 of the PDF):
I first presented this result to a recent convention of reliability and quality control statisticians working in the computer and aerospace industries; and at this point the meeting was thrown into an uproar, about a dozen people trying to shout me down at once. They told me, “This is complete nonsense. A method as firmly established and thoroughly worked over as confidence intervals can’t possibly do such a thing. You are maligning a very great man; Neyman would never have advocated a method that breaks down on such a simple problem. If you can’t do your arithmetic right, you have no business running around giving talks like this”.
After partial calm was restored, I went a second time, very slowly and carefully, through the numerical work [...] with all of them leering at me, eager to see who would be the first to catch my mistake [...] In the end they had to concede that my result was correct after all.
To make a long story short, my talk was extended to four hours (all afternoon), and their reaction finally changed to: “My God – why didn’t somebody tell me about these things before? My professors and textbooks never said anything about this. Now I have to go back home and recheck everything I’ve done for years.”
This incident makes an interesting commentary on the kind of indoctrination that teachers of orthodox statistics have been giving their students for two generations now.
Can you point to examples of these “holy wars”? I haven’t encountered something I’d describe like that, so I don’t know if we’ve been seeing different things, or just interpreting it differently.
To me it looks like a tension between a method that’s theoretically better but not well-established, and a method that is not ideal but more widely understood so more convenient—a bit like the tension between the metric and imperial systems, or between flash and html5.
The term “holy war” or “religious war” is often used to describe debates where people advocate for a side with an intensity disproportionate to the stakes, (e.g. the proper pronunciation of “gif”, vi vs. emacs, surrogate vs. natural primary keys in the RDBM). That’s how I read the OP, and it’s fitting in context.
Sure, I’m just not sure which debates he’s referring to … is it on LessWrong? Elsewhere?
[etc.]
Ugh. Here is a good heuristic:
“Not in stats or machine learning? Stop talking about this.”
Dude, I’m being genuinely curious about what “holy wars” he’s talking about. So far I got:
a definition of “holy war” in this context
a snotty “shut up, only statisticians are allowed to talk about this topic”
… but zero actual answers, so I can’t even tell if he’s talking about some stupid overblown bullshit, or if he’s just exaggerating what is actually a pretty low-key difference in opinion.
A “holy war” between Bayesians and frequentists exists in the modern academic literature for statistics, machine learning, econometrics, and philosophy (this is a non-exhaustive list).
Bradley Efron, who is arguably the most accomplished statistician alive, wrote the following in a commentary for Science in 2013 [1]:
In another paper published in 2013, Efron wrote [2]:
Thirty years ago, Efron was more critical of Bayesian statistics [3]:
The following bit of friendly banter in 1965 between M. S. Bartlett and John W. Pratt shows that the holy war was ongoing 50 years ago [4]:
For further reading I recommend [5], [6], [7].
[1]: Efron, Bradley. 2013. “Bayes’ Theorem in the 21st Century.” Science 340 (6137) (June 7): 1177–1178. doi:10.1126/science.1236536.
[2]: Efron, Bradley. 2013. “A 250-Year Argument: Belief, Behavior, and the Bootstrap.” Bulletin of the American Mathematical Society 50 (1) (April 25): 129–146. doi:10.1090/S0273-0979-2012-01374-5.
[3]: Efron, B. 1986. “Why Isn’t Everyone a Bayesian?” American Statistician 40 (1) (February): 1–11. doi:10.1080/00031305.1986.10475342.
[4]: Pratt, John W. 1965. “Bayesian Interpretation of Standard Inference Statements.” Journal of the Royal Statistical Society: Series B (Methodological) 27 (2): 169–203. http://www.jstor.org/stable/2984190.
[5]: Senn, Stephen. 2011. “You May Believe You Are a Bayesian but You Are Probably Wrong.” Rationality, Markets and Morals 2: 48–66. http://www.rmm-journal.com/htdocs/volume2.html.
[6]: Gelman, Andrew. 2011. “Induction and Deduction in Bayesian Data Analysis.” Rationality, Markets and Morals 2: 67–78. http://www.rmm-journal.com/htdocs/volume2.html.
[7]: Gelman, Andrew, and Christian P. Robert. 2012. “‘Not Only Defended but Also Applied’: The Perceived Absurdity of Bayesian Inference”. Statistics; Theory. arXiv (June 28).
Ilya responded to your second paragraph not the first one. metric vs. imperial or flash vs. html5 are not good analogies.
For lots of “holy war” anecdotes, see The Theory That Would Not Die by Sharon Bertsch McGrayne.
Do you consider personal insults, accusations of fraud, or splitting academic departments along party lines to be “a pretty low-key difference in opinion”? If so, then it is “overblown bullshit,” otherwise it isn’t.
Various bits of Jaynes’s “Confidence intervals vs Bayesian intervals” seem holy war-ish to me. Perhaps the juiciest bit (from pages 197-198, or pages 23-24 of the PDF):