In my mind I definitely hold romantic notions of pure-and-perfect scientists, whose work makes crisp predictions we can all see and that changes everyone’s minds. Yet it appears that Kuhn paints a far messier picture of science, to the extent that at no point does anyone, really, know what they’re doing, and that their high-level models shouldn’t be trusted to reach sane conclusions there (nor should I trust myself).
I update significantly toward the not-trusting-myself position reading this. I update downward on us getting AGI right the first time (i.e. building a paradigm that will produce aligned AGI that we trust in and that we’re well-calibrated about that trust). I also increase my desire to study the history of science, math, philosophy, and knowledge.
Remember that Eliezer’s version of Science vs Bayes is itself a paradigm. IMO it meshes imperfectly with Kuhn’s ideas as Scott presents them in this post.
This post makes a much stronger case for the anti-Bayes side in Eliezer’s “Science Doesn’t Trust Your Rationality”.
In my mind I definitely hold romantic notions of pure-and-perfect scientists, whose work makes crisp predictions we can all see and that changes everyone’s minds. Yet it appears that Kuhn paints a far messier picture of science, to the extent that at no point does anyone, really, know what they’re doing, and that their high-level models shouldn’t be trusted to reach sane conclusions there (nor should I trust myself).
I update significantly toward the not-trusting-myself position reading this. I update downward on us getting AGI right the first time (i.e. building a paradigm that will produce aligned AGI that we trust in and that we’re well-calibrated about that trust). I also increase my desire to study the history of science, math, philosophy, and knowledge.
Remember that Eliezer’s version of Science vs Bayes is itself a paradigm. IMO it meshes imperfectly with Kuhn’s ideas as Scott presents them in this post.