Today I watched a friend do calibration practice and was reminded of how wide you have to cast your net to get well-calibrated 90% confidence. This is true even when the questions aren’t gotchas, just because you won’t think of all the ways something could be wildly unlike your quick estimate’s model. Being well-calibrated for 90% confidence intervals (even though this doesn’t sound all that confident!) requires giving lots of room even in questions you really do know pretty well, because you will feel like you really do know pretty well when in fact you’re missing something that wrecks you by an order of magnitude.
Being miscalibrated can feel like “if it were outside of this range, I have just… no explanation for that”—and then it turns out there’s a completely reasonable explanation.
Anyway, I thought a fun exercise would be describing weird situations we’ve encountered that turned out to have reasonable explanations. In initial descriptions, present only the information you (and whoever was thinking it over with you) remembered to consider at the time, then follow up in ROT-13 with what made the actual sequence of events come clear.