This is part 9 of 30 in the Hammertime Sequence. Click here for the intro.
I’ve been thinking about whether or not regular betting, prediction markets, and being well-calibrated is actually useful, and if so how to practice train calibration on a short feedback loop.
Being able to make accurate time-to-completion estimates, at least, is extremely powerful. This post describes my current strategy for staying calibrated about time estimates.
Day 9: Time Calibration
Of all the cognitive biases in the Sequences, Planning fallacy seems to be one of the most directly harmful and eminently fixable. The goal of today’s exercise is to build a tool for routinely checking your calibration about how long things take.
Although Planning fallacy is the clear antagonist in this situation, I also want to gesture at a second class of failures I’ve been facing which involve systematically overestimating the difficulty of things.
Vortices of Dread
After spending a few days checking my calibration, I was surprised at the sheer number of things I routinely overestimate the difficulty of, mostly because of an ingrained fear of housework and the bureaucratic machine.
Several years ago, I watched my father spend nearly a full week on taxes, reading all the fine print, cross-referencing internet forums, and triple-checking every field. I did my own taxes for the first time last year, expecting it to only be more nightmarish – after all, my father’s experience surely saved him mounds of time already, right? Instead, the whole process took me a single afternoon.
Two weeks ago, I set out to get a travel visa done with a travel agent after months of dread. I blocked off the entire afternoon in the (it seemed to me) likely event that I’d have to drive back and forth to pick up, print, and/or fix documentation. The visit ended up taking a total of ten minutes, not counting the two mile drive.
Last week, I set out on an odyssey to make a photo album for family members, dreading the many evenings I would pore over old files and spend arranging prints. The whole process took two and a half hours from start to finish with the timely aid of Yoda Timers.
What threw my award-winning calibration off so wildly? Two things were at play:
First, most of my System 1 data on how long things take comes from watching my anal-retentive parents. I instinctively feel that cooking a meal takes nearly an hour, that every field on every form needs to be checked twice by every individual involved, that you should always arrive fifteen minutes early, and that the bureaucratic machine is constantly out to Get You. That gave me the opposite of Planning fallacy.
Second, the creeping dread around problems became a self-fulfilling prophecy. Even though I was relieved after finishing my taxes in one afternoon, my memory of the experience is still dominated by the weeks of slowly building anxiety leading up to the event. In contrast, I hardly remember the actual filling out of forms. I suspect System 1 was picking up these awful weeks as signal of doom.
Calibration Challenge
In the next day, keep an eye out for clear-cut work and train your calibration on how long things take. Make a System 1 prediction about how long each activity takes, set a Yoda Timer for that amount of time, and try to hit that time. If you expect something to take longer than an hour, break it up into clearly-demarcated chunks to calibrate individually.
(There is, of course, the confounding factor of the timer, but if you find yourself significantly more efficient with the timer goading you on … maybe that’s something to consider doing regularly.)
If you’re anything like me, you’ll be wildly surprised by how systematically wrong your models are, in at least one direction. If surprised, update!
Daily Challenge
Share your worst case of Planning fallacy.
My worst case of the planning fallacy is a very general one—any assignment with a deadline. I consistently think “this problem set isn’t so hard, I’ll be able to get it done on Tuesday even though it’s due Friday.” In reality, I’m always working right up until the deadline, pretty much regardless of how hard the assignment is. It’s even worse at work, where a large fraction of the deadlines are made up and easy to slip past without anyone getting mad. I think the underlying dynamics here aren’t just about the planning fallacy, it’s also a matter of how my personal control systems operate, and I’m trying to address both pieces as part of hammertime.
I’m rather prone to both the Planning Fallacy and what you call vortices of dread due to overestimation of time/difficulty. I do indeed benefit from trying to solve the latter with timers and such.
For example, I used to always take forever to get ready to go to bed, because as I went about the house doing all the related tasks I had my phone with me to listen to podcasts because getting ready for bed was so boring I couldn’t do it without some entertainment, and since I had my phone with me I kept getting distracted by things on it. One week, I decided to make a rule (importantly, a temporary one, only for that week! otherwise I would’ve been too scared to institute it) that I would set an alarm each day at 11:15 pm, and when that alarm rang I had to put my phone down and get ready for bed and not touch my phone until I was in bed already. The alarm was set a full 45 minutes before the time I intended to actually go to sleep because I expected that getting ready for bed would take up a big chunk of that time—but I found that actually, if I do all the tasks without procrastinating, they take no more than about ten minutes, which is in fact a pretty tolerable amount of time to be bored for. So now that feels doable and getting ready to bed no longer takes forever for me.
Similarly, during the aversion factoring exercise a couple days ago, one of the things I thought about was that I procrastinate hugely on emptying the trash cans in my room and in the bathroom, which can get pretty gross. I arrived at the thought that one way I can make it less scary is by setting a five-minute timer while I do it, because probably it will only take about five minutes or not much longer, but time doesn’t feel real to me so it’s hard to benefit from the fact that it’s a very short task unless I make that extremely salient? Anyway, I set the timer and I did the task and it actually only took about three and a half minutes, which was even less than I expected. I predict that in the future I will take out this trash more promptly.
A thing I’ve been thinking about doing but haven’t yet gotten around to is calibration training on whether I will make it to various plans I make, and if so how late I will be to them. I’m very bad at punctuality and also somewhat flakey due to low energy and brainstuff; actually improving this seems too hard to tackle right now (and is not my highest priority), so I’ve been trying to get into the habit of warning people that I might not make it to things, and ideally give a probability estimate of my making it. But I realized I actually don’t have a good way of knowing what the probability is, beyond a felt sense which I assume is better than chance but probably not quite correct. So I want to make some predictions about it and see how I do. Haven’t done it yet, though.
I’ll try the calibration on predictions of how long individual tasks will take, too, though not today, as today is not a particularly worklike day.
Well, what comes to mind for me is a little tangential, but I suppose I’ll share it anyhow. I had a lack-of-planning experience at my undergrad: I had done a research project over a semester, and was under the impression I’d be turning in a brief set of remarks and conclusions. Nope—full five-page report, complete with figures. I found this out five hours before it was due.
Amazingly, I got a B+ on this from a very strict prof; I was indeed proud of the report produced (just not of how it came about). In any case, I am much more methodical now.
I’m not certain if this qualifies as a planning fallacy, but I’ve noticed a class or problem where a large nebulous task isn’t made actionable, and we just expect it to happen at some point. More an error of “when it will be done by” than “how long it will take.”
For example, my family knew for maybe a year that we would benefit from an exercise machine, and had discussed it many times. It was only when I realized the problem and set a deadline for myself that we actually got it.
“Yeah, I can totally do my master thesis in six months, even it if involves examining a large database of newspaper articles by myself, inventing a methodology to analyse them that translates in quantitative data, invent an observation grid for what people would usually treat as subjective evaluations, mapping and quantifying the business relationships between newspapers and other industries, and generally pushing past the methodology limits that prevented studies I saw so far to actually prove quantitatively that there were in fact a relationship between newspaper relationships with fossil fuels industries and their treatment of climate change in the news, while I know nothing about journalism studies or text analysis. No, my tendency to procrastinate hard or unpleasant things I don’t know how to do won’t be a problem. Why do you ask?”
It took a bit less than one and a half years.
My worst case of planning fallacy was when I thought I’d just come back to work after the maternity leave. I did get a job, then another, and then another, but I’ve never actually come back.
The “creeping dread” you mentioned is why Structured Procrastination doesn’t work for me, even if it helps me get more things done. The feeling of dread and guilt I get can be so strong that it takes over my whole experience. Maybe it would work for me if I was self-employed—i.e, if my number one task was just a personal project. (and maybe the author addresses this—I haven’t read too much into it).
Worst case of planning fallacy
Not serious, but recent: My dad asked for help replacing a shutter on the side of the house—it would take “only 15 − 20 minutes tops”. I factored in the planning fallacy and expected it to take an hour. It took 3 hours. Hofstadter’s Law strikes again.
I’ve noticed I’m too pessimistic on the micro but too optimistic on the macro. I’ll estimate that a small task on a song (like quantizing the drums) will take 4 hours and finishing the song itself will take like 2 weeks. When I actually quantize the drums, it only takes 45 minutes. Yet finishing the song takes 2 months, even though it’s made up of small tasks that don’t take as much time as I thought. Most of my time wasted tends to come from avoiding doing those small tasks. The avoidance itself might come from feeling like these small tasks take forever, even though I have a lot of evidence that it doesn’t.
I’ve mostly been aware of the planning fallacy and how despite knowing of it for many years it still often affects me (mostly for things where I simply lack the awareness of realizing that the planning fallacy would play a role at all; so not so much for big projects, but rather for things that I never really focus on explicitly, such as overhead when getting somewhere). The second category you mention however is something I too experience frequently, but having lacked a term (/model) for it, I didn’t really think about it as a thing.
I wonder what classes of problems typically fall into the different categories. At first I thought it may simply depend on whether I feel positive or negative about a task (positive → overly optimistic → planning fallacy; negative → pessimistic → vortex of dread), but the “overhead when getting somewhere” example doesn’t really fit the theory, and also one typical example for the planning fallacy is students having to hand in an assignment by a certain date, which usually is more on the negative side. But I guess the resolution to this is simply that the vortex of dread is not different from the planning fallacy, but a frequent cause of it.
we tend to overestimate how long things take that we feel negative about → vortex of dread
this causes us to procrastinate it more than we otherwise would → planning fallacy (so the “net time” is lower than anticipated, but the total time until completion is longer than anticipated)
Which leaves me with three scenarios:
positive things → planning fallacy due to optimism
negative things → vortex of dread → planning fallacy due to procrastination
trivial things I fail to explicitly think about → planning fallacy due to ignorance/negligence
And thus there may be different approaches to solving each of them, such as
pre-mortem / murphyjitsu, outside view
knowing about the vortex of dread concept, yoda timers, scheduling, intentionality
TAPs I guess?