Excusing a Failure to Adjust
Crossposted from Figuring Figuring.
Here is a handle for a mistake I make all the time that works really well for me… that is the handle works really well for me. I make this mistake fairly often, but of course I notice it in other people more often.
Imagine that your credence in a belief is represented by a physical slider. When you learn a new piece of evidence or argument or whatever, there are two kinds of mental motions you can react with. You can adjust the slider up or down, or you can explain why you don’t have to adjust the slider at all.
Sometimes the right thing to do really is to not adjust the slider. Sometimes people will think that you should adjust the slider when really you shouldn’t for reasons that are difficult to explain. But this is far far rarer than cases where you should adjust the slider, but instead you focus on coming up with reasons to not adjust the slider. On some days, it seems like 90% of human cognition is coming up with reasons to not adjust the slider.
And of course, for beliefs that you like, you will come up with excuses for refusing to adjust the slider down way more often than you will come up with excuses for refusing to adjust the slider up. And vice versa for beliefs that you dislike.
Probably, you would be better off banning that mental motion altogether. Nearly every new piece of evidence should cause some adjustment, even if it is very small. Most conversations should be about how much to adjust, not about whether to adjust. But banning that mental motion is difficult, and before you can, you must learn to notice the motion, and learn how it feels different from other motions.
This is the best tool I have found for noticing so far. I ask myself: “Am I trying to explain why I don’t need to adjust the slider?” Something different might work better for you, but I highly recommend trying at all to notice the difference.
I’m concerned that rationalist values can promote excessive sliding, just because that seems like the virtuous thing to do, when really much of the new evidence we receive is weak or inconclusive (at least at the time we recieve it). Looking to adjust your slider after every conversation seems to be setting yourself up for error, to me, because I don’t think it’s realistic to think we can discriminate that well based on evidence that would otherwise be so easy to dismiss as it trickles in. Although, for many beliefs, I suppose there isn’t much reason to privilege your previous level of credence over what you feel after getting slightly more ambiguous evidence...
More generally, there’s a difference between things being true and being useful. Believing that sometimes you should not update isn’t a really useful habit as it forces the rationalizations you mentioned.
Another example is believing “willpower is a limited quantity” vs. “it’s a muscle and the more I use it the stronger I get”. The first belief will push you towards not doing anything, which is similar to the default mode of not updating in your story.