I wish more people 1. tried practicing the skills and techniques they think are important as rationalists and 2. reported back on how it went. Thank you Olli for doing so and writing up what happened!
Being well calibrated is something I aspire to, and so the advice on particular places where one might stumble (pointing out the >90% region is difficult, pointing out that ones gut may get anchored on a particular percentage for no good reason, pointing out switching domains threw things off for a little) is helpful. I’m a little nervous about how changing question category apparently lead to poorer calibration for a while. It makes sense why that would be the case, but my ideal art of rationality would work well across domains. Otherwise, why not study that particular domain more? I do like the application to day-to-day problems; “do I have peanut butter at home or did I run out?” is the kind of thing I run into on at least a daily basis.
I’d love to have a dozen such reports from a dozen people’s attempts, both to see if a pattern stood out of where common mistakes are (“Be cautious, Laplace’s Rule works a bit differently when there can be multiple outcomes”) and to get more datapoints that practice works. That’s not a knock against what Olli’s written here, that’s a wish for more people to follow up and do this! Without feedback on what techniques work and what it looks like to improve, building a martial art of rationality gets much harder. With feedback like this, other people can better understand what’s worth practicing and what’s realistic to expect.
That’s the most important takeaway I had from this takeaway. The repeated practice worked, and Olli got more calibrated as they practiced.
I’m inclined to think the Best Of LessWrong posts should include, not just the big insights or the shiny new techniques, but the dutiful reports years later about how those techniques have impacts on normal life. I’d like to lightly recommend Takeaways From Calibration Training for inclusion in the Best Of LessWrong Posts.
I wish more people 1. tried practicing the skills and techniques they think are important as rationalists and 2. reported back on how it went. Thank you Olli for doing so and writing up what happened!
Being well calibrated is something I aspire to, and so the advice on particular places where one might stumble (pointing out the >90% region is difficult, pointing out that ones gut may get anchored on a particular percentage for no good reason, pointing out switching domains threw things off for a little) is helpful. I’m a little nervous about how changing question category apparently lead to poorer calibration for a while. It makes sense why that would be the case, but my ideal art of rationality would work well across domains. Otherwise, why not study that particular domain more? I do like the application to day-to-day problems; “do I have peanut butter at home or did I run out?” is the kind of thing I run into on at least a daily basis.
I’d love to have a dozen such reports from a dozen people’s attempts, both to see if a pattern stood out of where common mistakes are (“Be cautious, Laplace’s Rule works a bit differently when there can be multiple outcomes”) and to get more datapoints that practice works. That’s not a knock against what Olli’s written here, that’s a wish for more people to follow up and do this! Without feedback on what techniques work and what it looks like to improve, building a martial art of rationality gets much harder. With feedback like this, other people can better understand what’s worth practicing and what’s realistic to expect.
That’s the most important takeaway I had from this takeaway. The repeated practice worked, and Olli got more calibrated as they practiced.
I’m inclined to think the Best Of LessWrong posts should include, not just the big insights or the shiny new techniques, but the dutiful reports years later about how those techniques have impacts on normal life. I’d like to lightly recommend Takeaways From Calibration Training for inclusion in the Best Of LessWrong Posts.