Epistemic status: Reference. Highly confident. I rely on what is presented here extensively in my own thinking.

Plans are recursive. Any plan can be decomposed into parts and those parts can be in turn be decomposed into further parts and so on until it is senseless to decompose any further. This is not a profound point, it is something we all appreciate intuitively and explicitly to at least some degree or another.

Nonetheless, the recursive nature is of such fundamental importance to the entire practice of planning that it warrants an explicit treatment.

Crucial principles of good planning are readily derived from a recursive model of planning. Having a solid grasp of the recursive models makes it harder to forget these key principles.
An explicit treatment of a topic can help it sink in deeper to intuition even if one already has some sense of it.
Even if everyone has an intuitive sense of something, it can be hard to talk about the thing if there isn’t a common explicit handle.

A quick definition of planning as used in this post: Planning is the selection of actions to be deliberately executed in order to achieve a desired outcome/goal. [1]

Plans are recursive

Usually, we treat plans in a very linear fashion. Step 1, Step 2, Step 3. To-Do lists capture this, and we can also draw an accompany thing diagram.

Example Linear Plan: Hosting a Dinner Party

Invite guests
Clean house
Prepare food

Linear diagram of a plan to host a dinner party

Of course, we all recognize that the three steps of the plan above are composed of further sub-steps. Nested to-do lists capture this well.

Example Recursive Plan: Hosting a Dinner Plan Expanded

Invite guests
1. Select date and time
2. Select guests
3. Issue invitations
Clean house
1. Call in favor from housemate
Prepare food
1. Buy ingredients
2. Cook food

These steps could be further decomposed, but this suffices to demonstrate the idea. We can draw another diagram.

Recursive tree diagram of a plan to host a dinner party

Why call it recursion?

Sometimes people use fancy technical terms in order to sound smart even though a more commonplace word would suffice. Arguably, I could have said that plans are hierarchical or nested or something. Why use the mathematics and computer science term, recursion?

I defend this choice with reference to the defining features of recursion:

You solve a problem by breaking it into smaller instances of the same type.
Each time you decompose the problem into a smaller problem, you apply the same function/procedure to the smaller piece as you did the larger one.
You continue decomposing until you reach a base case which requires no further decomposition.
The overall problem is solved by rolling up all the solutions to lower-level decomposed sub-problems.

[The calculation of factorials is the canonical introductory example for recursion and neatly illustrates all of these features.]

Planning has all of these features and is therefore rightly described as being recursive.

We see that plans can be decomposed into sub-plans which might then be decomposed into further sub-plans. Eventually you stop, i.e. you reach a base case. When is this? It depends on the context, but one heuristic is to stop when the level is so trivial that it doesn’t require further decomposition. In the above example of hosting a dinner party, I might not need to recurse to the level of planning out the steps of sending an emails. I can take for granted that I know how to carry that out.

The Core Planning Process

What is the universal planning function/procedure in the context of planning? A decision theorist or AI expert might be able to offer a little more rigor here, but the universal core planning process we repeatedly apply whenever we are planning is going to look roughly like this:

Enumerate the set of possible actions to take.
Predict which actions will result in which outcomes (and with which likelihood).
Assign relative preferences to each of the potential outcomes.
Use the above points to prioritize the actions you will take based on the expected costs, benefits, and risks associated

How the human brain executes the above steps is complicated. And what is required to execute those steps will also differ dramatically between plans and at different levels within a plan. In particular, sub-plans can require very different models to map actions to outcomes than the models used at a higher level. The models used for business strategy are different from those used to file taxes, though both are part of the broader “business success” goal. Still, we shouldn’t let the variation between plans mask that the core steps are always the same.

Plans are Two-Fold Recursive

We can actually say that planning is doubly recursive. Planning is both recursive in planning, i.e. when you’re choosing what to do [2], and also in execution. To execute a plan, you must execute each of each the sub-plans, and each of the sub-plans’ sub-plans and so on. When we’re talking about execution, the recursive function to be applied at each level is simply “Do X”.

Implications of the Recursive Nature

The following section details some key principles of good planning that can be directly derived from the recursive nature of plans. Even though both the recursive nature and the principles presented here seem sensible, if not obvious, on their own, linking the two together makes it clear why these planning principles are necessarily so.

Goals/Steps/Actions/Plans are Isomorphic

When we plan, we usually start with a goal. Then we figure out actions and sub-plans we will take towards the goal. This picture is somewhat misleading. It makes it sound like the goal and steps we take towards it are of a different kind, and so encourages us to treat them differently. Usually, that means treating the goal as more fixed than it ought to be.

Someone might decide that they have the goal of becoming a lawyer. They start of with this goal and start evaluating steps they might take towards achieving it. Maybe they start thinking about which law school they want to go, where they will get a student loan, how they’re going to improve their study habits. They start considering the obstacles and how to overcome them.

For each possible step towards the goal, they enumerate and evaluate the options for their benefits, costs, and risks. What is so easily missed is whether or not the “goal” was the right step to begin with. One must remember that the goal itself is likely only a step within some yet larger step. The goal is a step too because, from the perspective of the recursive tree, all the nodes are of the same type. [Perhaps we wish to treat nodes at the very top or very bottom differently, but otherwise, there’s no difference.]

It is only in our minds that we distinguish between goal, step, action, and plan. Structurally they are the same thing.

See? They’re all just plans.

The exception to the isomorphism are terminal goals which we wish to accomplish purely for their own sake rather than instrumental goals which we seek to accomplish to attain something else. Terminal goals are not steps within another plan and so the isomorphism does not hold with them. Still, I claim that since the overwhelming majority of our goals and plans are only instrumental, it is reasonable to assert that goals (generically) are isomorphic to plans. Most nodes in the recursive tree are in the middle. It’s true enough that it’s helpful to remember it as such.

Almost All Plans are Sub-Plans; Almost All Goals are Sub-Goals

The corollary to the above point that goals/steps/actions/plans are all the same is that [almost] all plans are sub-plans and [almost] all goals are sub-goals. If you have a goal, it’s probably because there’s something you hope that goal will accomplish.

We can reuse the lawyer example from above.

What can be lost when you start with your goal and don’t look any higher is all the reasons you selected your goal in the first place. It is worth remembering and examining those reasons. Possibly you selected those reasons long ago and they no longer apply. Possibly it was an intuitive or emotional decision but not an especially good one. If you don’t think about the reasons behind your goals, i.e., your higher-level goals, then you very likely will either select the wrong goals or select the right approximate goal which you then execute in such a way as to not accomplish your higher-level goals.

Peter Thiel, famous billionaire investor, went to Stanford law school and practiced in a prestigious New York law firm for seven months and three days. He is oft quoted as describing that this law firm was a place where “from the outside everyone wanted to get in and on the inside everybody wanted to get outside.” When he left, a colleague told Thiel that he didn’t know it was “possible to leave Alcatraz.” Clearly not somewhere he actually wanted to be. Peter Thiel cites his reason for going to law school in the first place: “So I think I got a JD because of the abstract prestige, because I didn’t know what else to do, and I didn’t concretely think through what it would be like to be a lawyer.”

Peter Thiel’s example highlights an important point. If you know why you have a particular goal, you can list out the assumptions and beliefs that cause you to have it. Once listed, the assumptions and beliefs can be examined, further evidence can be collected, and you can avoid making plans based on shoddy beliefs and limited information. You can ensure that you think through concretely what it would be like to be a lawyer, or as Thiel advises others, go identify someone with a JD who’s a good role model.

The elucidation and examination of assumptions should be part of your “core planning process” which should carried out appropriately at each level and node of your plan.

In a sense, you need to dissolve the distinction between plan/goal/action/step and see the recursive tree of your plan for what it is. Then you apply your planning process to the nodes indiscriminately. When starting with a goal, you ought to “recurse upwards” until you identify higher-level goals that you are sufficiently confident in [3]. Even if you are completely certain about your higher-level goals, you still want to be thinking about them because their details dictate what the details of your lower-level goals need to be.

When our lawyer-wannabe sits down to plan how they’re going to become a lawyer, they will make worse choices if they forget what the point of becoming a lawyer was. The point of a lawyer is not to become a lawyer inherently, more likely it’s to have a good job with a high salary, prestige, and satisfying work as part of overall having a good life. You can avoid repeating Peter Thiel’s mistake.

If you remember why you care about a goal, you can examine whether this goal will actually achieve your higher level goals. The lawyer-wannabe can look up lawyer job opening and salaries, find lawyers she can question about their lifestyle, poll some friends to see if they actually respect lawyers, and so on. She might find out being a lawyer doesn’t serve her higher-level goals and should maybe pursue something else. It’d save her a lot of time to find that out sooner rather than later.
It’s possible that there are versions of becoming a lawyer which meet the higher level goals and versions which don’t. Becoming a barrister might have a lot of prestige and high salary yet be stressful and have extremely long hours. Conversely, being an environmental lawyer might pay less but an overall better lifestyle. Keeping the true goal in mind helps determine the details you want to target.

I fear that the very particular obvious career example I have presented won’t translate readily in the minds of readers to all the other cases where this applies. It applies to projects within startups and business, it applies to the books you read, it applies to the people you date and how you find them, it applies to your hobbies and your vacation. It applies to any planning of non-trivial complexity. I encourage the reader to search for a few examples from their own life where they might benefit from applying this line of thinking.

A goal is only right if it actually serves your higher-level goals

If you didn’t pick the right goal to begin with, then no matter how hard you hit that goal, all is for naught. In terms of a recursive trees, for a goal to matter, there needs to be an “unbroken path” to something you terminally value.

If the lawyer-wannabe succeeds super hard at becoming a lawyer: graduates Summa Cum Laude from the best school, gets a role in the most prestigious firm, etc., it doesn’t matter at all if it turns out being a lawyer was the wrong choice.

To reword points from above, goals can be wrong in two ways:

You picked the wrong goal entirely. Another option was entirely better.
You picked roughly the right goal, but you implemented it such that it had no value.

An aside about long-term planning and feedback

High-level goal selection is often fraught because there is slow feedback. A young person might not discover for years that they didn’t actually want to be a lawyer, so it’s easy to pick the wrong high-level goal and grind towards for years, only getting feedback that it was the wrong choice after they graduate and start working. Peter Thiel is an example. Another class of examples is startups and business deciding which products to build. Done wrong, a startup can pick a long-term goal and work for months or years before finding out it was the wrong goal.

Worse than the pure absence of quick-feedback, long-term planning is often influenced by short-term factors such as emotions. “Gah, planning is stressful! I’m just going to pick something and stick with it.” Rather than focus on the long-term goal, how the person feels in the moment is prioritized. Relatedly, as with Peter Thiel, instead of trying to pick the best long-term plan, someone might pick the plan which gets the greatest social reward (since there are indeed rewards for just having cool plans). There can faster feedback on whether your plan is cool then whether it will succeed. You’ll know as soon as you boast to your friends or describe your product plans to your startup’s investors. The quick feedback on emotions and social rewards often outweighs the value of a long-term goal one is supposedly planning towards. Beware.

The precise success criteria of a lower-level goal are determined by the higher-level goals

The importance of being clear on why a goal has been selected is never clearer than in the case of delegation. A manager asks a research engineer to write a report on their recent research. The engineer does so, but the manager is not pleased! The report is a technical, suitable only for other engineers, yet this is not what the manager wanted—the manager wanted a report to give to the business executives.

The research engineer delivered what was requested “a report on their recent research”, but since this didn’t connect with the higher level goal of having a way to update the executives, it was worthless.

The danger is that we humans often encapsulate our goals and plans very crudely and with very little detail. A few words like “become a lawyer.” We rely on ourselves and others implicitly understanding the specific versions of those goals which would count as success—yet so often we don’t.

Even when the goal is certain, you need to understand the bigger picture in order to be able to make lower-level implementation choices. Anyone delegating a task is advised to instruct the delegatee in why the task is being requested. The more detail the better.

You can’t just ask someone to book you a venue for an event. You either need to tell the exact kind of venue you want or tell them what kind of event it is so they know what kind is appropriate. And even if you say “fits 50 people, within X budget, within Y miles of location A” there could still be further decisions whose ideal selection is made only if you know why a venue is needed: what kind of event, which guests and therefore which ambiance and so on would best.

Delegation is hard because it means splitting the recursive tree among multiple people. In the case of principal-agent problems, you’re transplanting a goal from one person’s tree to another. You can see why that’d run into trouble.

Final example: as a Data Scientist, I was often asked to build things with little specification and no explanation of why. A technical example here is that if you don’t understand how a classifier will be used, then even if you can build one, you don’t know how to make a dozen relevant design choices, e.g. setting thresholds which determine false positives and false negatives. Building something without an idea of why is a great recipe for building a version of it which doesn’t meet the true user requirements. [4]

Wrapping up

Though the ideas presented here are simple, I hope that if you have read this far, this post will have deepened your appreciation of this fundamental property of plans and will result in you making better plans towards good things.

Appendix: Multiple Goals

Though it was skipped for the sake of simplicity in the main text, realistically, many our plans are executed with more than one goal in mind. This is entirely compatible with a recursive model of plans, but it makes the diagrams messier. In order to not distract from the main points, examples in the text were chosen to roll-up neatly into a single goal.

Below is a realistic recursive plan diagram illustrating multiple goals. Makes you appreciate what your brain has to deal with.

Modern art: a recursive tree diagram of a plan where sub-plans feed into multiple higher-level plans, e.g. actions are taken towards multiple goals.

Endnotes

[1] This broad definition stands in opposition to a common definition which uses planning primarily in contexts of scheduling, e.g. plan your day, plan your week. The broader definition here is essentially synonymous with decision-making, perhaps differing only in connotation. Decisions somewhat imply a one-off choice between options whereas planning implies selection of multiple actions to be taken over time. The term planning also somewhat more than decision-making highlights that there is a goal one wishes to achieve.

[2] With the broad definition of planning used here, it’s important to note that planning needn’t happen all at once. More commonly, different components of a plan (selection of action) happen at different times. Very commonly, higher-levels of the plans are selected up front and lower-level actions are selected near or at the time of execution.

[3] This is often really, very hard and time-consuming. If you’re haven’t recursed explicitly upwards to determine what you want and what your true high-level goals are, this might take a while including both the time to do it and possibly the time spent gaining the skills do it. The good news is that after initial investment you get to reuse your knowledge of your high-level goals repeatedly when making lower-level plans such that you make much better low-level plans.

[4] My boast: determined to not build the wrong thing, I spent enough time figuring out what the why reasons for our engineering work should be that my company eventually asked me to take that over as my explicit role.

Plans are Recursive & Why This is Important