Instrumental Rationality: Postmortem
[In April, I set off to write a series of essays about instrumental rationality. Now that the project’s reached a pretty good stopping point, I’m looking back to see how my expectations and goals played out.]
Initial Goals:
[What I originally wanted the book to be like. My estimates vs reality]
Originally, I wanted to write something that would tie together all the current research on topics like motivation, planning, and habits. I felt like lots of LessWrong posts touched upon certain areas, like hyperbolic discounting, but there wasn’t a central place where it all came together.
I wanted a new central beacon to point people to when The Sequences didn’t quite fit.
I’d envisioned a sequence of essays which would give an overview of the latest developments in the field followed by concrete techniques, ala CFAR.
Here are some of the topics I’d originally wanted to cover:
Willpower
Attention
Habits
Behavioral Economics
Motivation
When I began this project, I had Planning 101 under my belt, which had taken me about 20 hours to complete. As far as base rates went, it seemed reasonable to think that the other topics would take a similar amount of time.
Looking back, it feels a little silly to think that Past Owen thought he could take on five more of those 20+ hour chunk projects. That would have easily been 100+ hours, in addition to editing, compiling, and a bunch of additional grunt work which snuck up on me.
(What was I thinking?)
In the end, I only managed to write one more primer—Habits 101—which ended up taking about twice as long, 40 hours, for just the actual writing portion. Reading up on the articles took up additional time, probably another 10 hours or more.
Though I didn’t complete my initial vision, I actually did fairly well on my own estimates:
I’d given myself an internal completion date of the beginning of September, and I actually finished around that time. (Hooray!)
I’d also estimated that I’d write about 10,000 words of new content for the book. And I went about 50% more than that, writing about 15,000 words of actual new content. (Hooray!)
(Although the counterfactual for the new content prediction isn’t as good as it seems because I probably would have written some of those blog posts regardless of whether or not I also held the intention to make a sequence.)
The Finished Result:
[Evaluating the end product, some things which didn’t make the final cut.]
So I ended up both finishing both on-target and on-time. How did the actual end product compare with my expectations?
Well, I’ve already pointed out how there was a lot of content I wanted to write which didn’t make it into the book. As for the content that did, here are the 9 sections that made it into the book:
Introduction
WTF Is Rationality?
Starting Advice
Planning 101
Interlude 1
Habits 101
Interlude 2
Attractor Theory
Closing Disclaimer
Of the 9 sections, I think that WTF Is Rationality?, Starting Advice, and Attractor Theory come closest to the sort of “crystallization” I’d originally hoped for.
In the time between the original blog posts and the polished book essays, I’d had time to try explaining them to people in person. I think the experience helped me understand which things were important to focus on and what background knowledge I needed to assume. The revision and slow iteration of ideas helped me figure out which components were the important ones to stress.
So I think those three sections turned out the best.
As for the two 101 primers, I think they were too bulky, and I think a better choice would have been to sacrifice depth for breadth. That is to say, cutting the length of Habits 101 in half (or even two-thirds) in order to make way for a short primer on both Attention and Behavioral Economics would have likely been good.
Otherwise, I think it feels a bit jarring to switch from a short heuristic-y essays about ways to approach life to a deep academic dive into psychology.
As the sequence currently stands, I think it’s unbalanced in terms of the topics it goes over. Partially due to the length of the two 101 primers, I think there’s an overemphasis on planning and habits.
Overall, I think it pushes the “master yourself” mindset over the “become one with yourself” mindset too hard, which I ended up writing about in the Closing Disclaimer.
Perhaps unfortunately, I’m also fairly confident that most people who read it won’t get much benefit out of it. (More on this in Evaluating Impact.)
Starting out, I also had some big hopes for the actual format of the book.
I think that most writing doesn’t do a good job of helping the reader chunk the information. That is to say, the information trying to be conveyed is often much clearer in the author’s mind than in the readers’. I claim this is often because the author has their own way of mentally structuring the information which they neglect to share during the actual writing.
Frustrated with this, I’d originally wanted to include several design features for the sequence to help with understanding:
Suggested Exercises at the end of every chapter, focused on developing practical skills.
Unique formatting, perhaps a different font color, for paragraphs which contained examples (to visually differentiate them from the rest of the text).
Visual outlines of how points connected in each essay. (You can see some simple prototype attempts in Habits 101.)
In the end, the only one that made it to the final book are the bracketed summaries in italics that precede each section.
I think that conveying ideas in general is actually rather quite difficult. Cooperation is required on both the part of the author and the reader, and even an engaging writing style (or cool design tricks) can only do so much.
Writing Time:
[Reflections on the actual writing process.]
I ended up having less time to write the entire book than I originally thought.
No surprise there.
(“You don’t plan for disaster. Disaster plans for you.”)
Part of the reason was because I was physically unavailable on account of my being at Google doing CS things for three weeks.
The other part was that I found myself mentally incapable of writing quality content for long bouts of time.
The roughly 100 hours I put into writing this book was scattered over about 4 months. That ends up being less than an hour a day, though the distribution of hours wasn’t uniform. When it came to writing this book, I found that certain times yielded far more productivity than others.
Also, looking back, it feels like the entire duration of time was necessary, even if most of it wasn’t actually spent writing. It feels like I just needed time in between writing sessions to let my mind do its thing under the hood, subconsciously. I’d write for about an hour, wander about the room (or do something else for an hour), and then I’d return to writing.
This seems like a specific instance of the general principle that breaks aren’t just a “fun” activity, but are actually a requisite for good work to happen.
So it was less about getting enough free time in a chunk, but more about getting enough of the right kinds of time, or something like that. For context, I think that one thing people overlook when considering making tradeoffs involving time is the nature of the time they’re gaining or losing.
For example, if you’re able to save ten minutes off your commute, that roughly translates to having ten additional minutes to spend at work, which might not be worth much. In contrast, if you’re able to extend your lunch break by twenty minutes, that could be enough time for a noontime nap, which might be very valuable.
Against the Incentive Gradient
[Writing longform has certain drawbacks.]
When I was working on this project, a friend pushed me to consider writing for a larger group like BuzzFeed or ClearerThinking. They argued that even though the type of content engagement I’d get from readers would be less, the net increase in audience size would mean that the aggregate impact would overall be larger.
The argument seemed good, but I ended up sticking to my original plan to write in longform.
(Longer exploration on this topic in The Best Self-Help Should Be Self-Defeating.)
Final Lessons:
[What were my concrete takeaways?]
There were several things I learned as a result of undertaking this project:
My internal estimates for my task completion time and writing rate are fairly well-calibrated.
I can make graphics of a quality I’m happy with at a rate of about 1 graphic an hour.
Experience and understanding of what research looks like. (EX: Filtering through papers to find promising things, writing summaries, simplifying at the right level, etc.)
Getting constant feedback is both important for my ability to continue projects, as well as improve the quality. I didn’t get enough of it this time around.
I had, overall, still underestimated the difficulty involved in writing and editing a book-style project.
The biggest one, though, was the burning question I had when setting out on this project: “Why the hell hasn’t anyone tried to make this type of freely available rationality handbook before?”
The answer, I think, looks something like this:
If you’re writing self-help content, the question of who the audience is inevitable. After all, at some point, someone’s supposed to be reading the content. And in my current understanding of people, it seems like there’s roughly two categories of readers (gross oversimplification alert!):
One, if you’re already smart and self-sufficient, then you’re already combing all the coolest blogs and books for insight. You’re a hyper-scholar who has either read and gotten value out of what I have to offer, or you’ve already got even more sophisticated models.
Two, if you’re the sort of insight junkie who’s looking for the next 5 minute article on how to restructure your workflow, then you may miss out on the actual good stuff. If you’re always only looking at the small insights, then the ontological lens of rationality (which is what I really want to illustrate), is largely lost.
So I guess one strong reason for why the sort of rationality handbook I was envisioning hasn’t happened is because it doesn’t really help out either of the groups.
The target audience is someone who’s a little mixed up in their stages of intellectual development.
I’m reminded of how I felt after talking about the book Gödel, Escher, Bach with some friends studying computer science. It’s a book which I very much enjoyed. However, none of them had been impressed with it, and they found it largely pedantic, taking too long to get to some basic insights.
Most people who read GEB, I suspect, either get lost in the dizzying array of cultural references and come out with a messy interpretation, or are already with maths and computer science such that the key points fall flat.
When I read it, I was in this weird spot where I didn’t know any computer science, but I also looked past the references, and I found something exciting that encouraged me to dive deeper.
++++
With this instrumental rationality sequence, I’d wanted to write something for other people to benefit from. In a funny sort of way, though, I guess I really did just end up writing a book for myself.
Massive props for completing the project, as well as following it up with this postmortem.
There’s a weird problem (which I don’t think you were directly approaching, but which I think is relevant) which is that people will often say “we need to have a version of the Sequences that doesn’t have all the weird Eliezer-isms”. And I think the weird Eliezerisms are similar to the Godel Escher Bachisms, and that what they do is grab some random subsection of the population and strongly appeal to them enough to make them take the thing seriously.
I touched on this a few posts ago, but I think what this series was missing was some kind of weird lifelonglearner-isms that turn off some people but are captivating enough that some people go “woah, this is cool and worth delving into.”
Cool, that definitely makes sense! I think I ended up biasing towards “giving information cleanly” and this sort of framing made me think less about “how can I embed this series with the sort of things that are idiosynchratic to me?”
And this of course ties back into things about having a more distinctive writing style / how that leads people in.
It’s very late feedback, but I think this was easily in the top three of the most well written self-help advice I’ve ever read.
It’s early to judge, but I also think it’s been one of the most useful. I have been kinda doing stuff similar to some of what you describe for a while, and I kept doing it because it was effective, and in the few days I had to try it having a more… explicit and systematic model of what actually works to shape my behaviour turned out to be useful.
So far I haven’t tried to apply it to too much stuff at once for fear of burning out, but problems I tried to fix in the past have been fixed for days now, where past solutions worked for mere hours before.
With the disclaimer you’ve made me realise that lately I had cut all of a sudden a lot of the stuff that makes me happy to improve productivity on stuff that’s important, which explains why my mood got lower. I will act to restore some balance into that as well.
Thanks for the feedback Emiya! I hope it ends up being useful for helping you get what you want to get done, done.
I never got the chance to update here, but I cleaned up some of the essays in the years since writing this series.
They can now be found here. Of note is that I massively edited Habits 101, and I think it now reads a lot tighter than before.
Thanks, I’ll check them out as well!
“Also, looking back, it feels like the entire duration of time was necessary, even if most of it wasn’t actually spent writing. It feels like I just needed time in between writing sessions to let my mind do its thing under the hood, subconsciously. I’d write for about an hour, wander about the room (or do something else for an hour), and then I’d return to writing.”
I find this to be the case with most mentally demanding tasks. I generally expect to only get as much as 4 hours of work done in a day because I’ll need to spend the rest of it on “breaks” where what I’m really doing is giving my brain space to consolidate memories and whatever else it needs to do to function at full capacity. It’s an important part of work that often goes overlooked because it’s not active, but for me it has been vitally important.
Seems to be one of the many ways mentally demanding work does not resemble more tedious labor.
“If you’re writing self-help content, the question of who the audience is inevitable. After all, at some point, someone’s supposed to be reading the content. And in my current understanding of people, it seems like there’s roughly two categories of readers (gross oversimplification alert!):
“One, if you’re already smart and self-sufficient, then you’re already combing all the coolest blogs and books for insight. You’re a hyper-scholar who has either read and gotten value out of what I have to offer, or you’ve already got even more sophisticated models.
“Two, if you’re the sort of insight junkie who’s looking for the next 5 minute article on how to restructure your workflow, then you may miss out on the actual good stuff. If you’re always only looking at the small insights, then the ontological lens of rationality (which is what I really want to illustrate), is largely lost.
“So I guess one strong reason for why the sort of rationality handbook I was envisioning hasn’t happened is because it doesn’t really help out either of the groups.”
At one point I tried to write a self-help book based on my reading of developmental psychology around operationalizing post-formal (adult) development. I worked up a bunch of fancy metaphors for explaining various things so they made intuitive sense and creating worksheets people could follow to help them along the way, but I ultimately abandoned the project for basically the audience reason you describe above.
If I was writing for people like me, then I by definition didn’t need what I was writing because I figured it out without it. In that case I need only keep linking to things to encourage people to check out the same ideas I had found so useful. If I was writing for people like the second ground you identify, those people seem mostly stuck and need to be pushed, but how you do this is highly individualized. So no matter what I wrote it would at best appeal to a very small audience, and so while maybe a worthy project was less clearly going to have a sufficiently impactful outcome for what I wanted to achieve.
My solution was to write for people like me about the new things I’m figuring out mostly and help people one-on-one in tailored ways that work for them since that seems more likely to me to produce the kinds of changes I would like to effect (I’d rather have a few, thorough, deep impacts than many partial, shallower impacts). Luckily for us it sounds like your preferences are leading you in a different direction where we’ll get to read some interesting intro content!
You could figure it out without it, but having the compiled notes of someone who’s already done that search can still save you a decent amount of time. Having external confirmation that your vague inkling is on the right track also makes you faster to un-vague it.