[Link]Rationalization is Superior to Rationality

Philosophy and the practice of Bayesian statistics

This is a 2012 paper by Andrew Gelman and Cosma Rohilla Shalizi on what they view as a misuse of Bayesian statistics in scientific reasoning. I found this interesting because their definition of hypothetico-deductivism closely matches up with Eliezer Yudkowsky’s definition of rationalization, and their definition of inductive inference closely matches up with his definition of rationality. The definitions:

Eliezer Yudkowsky:

Rationality—Starting from evidence, and then crunching probability flows, in order to output a probable conclusion.

Rationalization—Starting from a conclusion, and then crunching probability flows, in order to output evidence apparently favoring that conclusion.

Andrew Gelman and Cosma Rohilla Shalizi:

Inductive Inference—An accretion of evidence is summarized by a posterior distribution, and scientific process is associated with the rise and fall in the posterior probabilities of various models.

Hypothetico-Deductivism—Scientists devise hypotheses, deduce implications for observations from them, and test those implications. Scientific hypotheses can be rejected (i.e., falsified), but never really established or accepted in the same way.

Now, what’s interesting about the paper is that in contrast to Eliezer Yudkowsky’s view they argue that rationalization (hypothetico-deductivism) is the correct analytic method, and rationality as Eliezer Yudkowsky defined it is wrong. They make the following argument:

Social-scientific data analysis is especially salient for our purposes because there is general agreement that, in this domain, all models in use are wrong – not merely falsifiable, but actually false. With enough data – and often only a fairly moderate amount – any analyst could reject any model now in use to any desired level of confidence. Model fitting is nonetheless a valuable activity, and indeed the crux of data analysis. To understand why this is so, we need to examine how models are built, fitted, used and checked, and the effects of misspecification on models.

They also argue Popper made multiple errors, but that his fundamental view is closer to correct than Kuhn’s, and that correct science is about attempting to falsify hypotheses. They simply disagree with how Popper went about doing it.

Another interesting issue to me is that if you look at the main post Against Rationalization, Adirian and Vladimir_Nesov both suggested that both forms of analysis are acceptable, but TheAncientGeek was the only one who argued rationalization over rationality, and his comment received multiple downvotes. This also appears to me to have been a major concept central to many parts of the sequences. Andrew Gelman and Eliezer Yudkowsky had a bloggingheads.tv conversation together, b̶u̶̶t̶̶ ̶̶I̶̶’̶̶m̶̶ ̶̶n̶̶o̶̶t̶̶ ̶̶s̶̶u̶̶r̶̶e̶̶ ̶̶i̶̶f̶̶ ̶̶t̶̶h̶̶i̶̶s̶̶ ̶̶p̶̶a̶̶r̶̶t̶̶i̶̶c̶̶u̶̶l̶̶a̶̶r̶̶ ̶̶t̶̶o̶̶p̶̶i̶̶c̶̶ ̶̶e̶̶v̶̶e̶̶r̶̶ ̶̶c̶̶a̶̶m̶̶e̶̶ ̶̶u̶̶p̶̶.̶̶

Thoughts?

Edit—Andrew Gelman and Eliezer Yudkowsky discuss this issue at the end of the bloggingheads video. Click on “The difference between Eliezer and Nassim” for their take. I also fixed a link.