Reasonable Explanations

To­day I watched a friend do cal­ibra­tion prac­tice and was re­minded of how wide you have to cast your net to get well-cal­ibrated 90% con­fi­dence. This is true even when the ques­tions aren’t gotchas, just be­cause you won’t think of all the ways some­thing could be wildly un­like your quick es­ti­mate’s model. Be­ing well-cal­ibrated for 90% con­fi­dence in­ter­vals (even though this doesn’t sound all that con­fi­dent!) re­quires giv­ing lots of room even in ques­tions you re­ally do know pretty well, be­cause you will feel like you re­ally do know pretty well when in fact you’re miss­ing some­thing that wrecks you by an or­der of mag­ni­tude.

Be­ing mis­cal­ibrated can feel like “if it were out­side of this range, I have just… no ex­pla­na­tion for that”—and then it turns out there’s a com­pletely rea­son­able ex­pla­na­tion.

Any­way, I thought a fun ex­er­cise would be de­scribing weird situ­a­tions we’ve en­coun­tered that turned out to have rea­son­able ex­pla­na­tions. In ini­tial de­scrip­tions, pre­sent only the in­for­ma­tion you (and who­ever was think­ing it over with you) re­mem­bered to con­sider at the time, then fol­low up in ROT-13 with what made the ac­tual se­quence of events come clear.