Plans are Recursive & Why This is Important

Epistemic sta­tus: Refer­ence. Highly con­fi­dent. I rely on what is pre­sented here ex­ten­sively in my own think­ing.

Plans are re­cur­sive. Any plan can be de­com­posed into parts and those parts can be in turn be de­com­posed into fur­ther parts and so on un­til it is sense­less to de­com­pose any fur­ther. This is not a profound point, it is some­thing we all ap­pre­ci­ate in­tu­itively and ex­plic­itly to at least some de­gree or an­other.

Nonethe­less, the re­cur­sive na­ture is of such fun­da­men­tal im­por­tance to the en­tire prac­tice of plan­ning that it war­rants an ex­plicit treat­ment.

  • Cru­cial prin­ci­ples of good plan­ning are read­ily de­rived from a re­cur­sive model of plan­ning. Hav­ing a solid grasp of the re­cur­sive mod­els makes it harder to for­get these key prin­ci­ples.

  • An ex­plicit treat­ment of a topic can help it sink in deeper to in­tu­ition even if one already has some sense of it.

  • Even if ev­ery­one has an in­tu­itive sense of some­thing, it can be hard to talk about the thing if there isn’t a com­mon ex­plicit han­dle.

A quick defi­ni­tion of plan­ning as used in this post: Plan­ning is the se­lec­tion of ac­tions to be de­liber­ately ex­e­cuted in or­der to achieve a de­sired out­come/​goal. [1]

Plans are recursive

Usu­ally, we treat plans in a very lin­ear fash­ion. Step 1, Step 2, Step 3. To-Do lists cap­ture this, and we can also draw an ac­com­pany thing di­a­gram.

Ex­am­ple Lin­ear Plan: Host­ing a Din­ner Party

  1. In­vite guests

  2. Clean house

  3. Pre­pare food

Lin­ear di­a­gram of a plan to host a din­ner party

Of course, we all rec­og­nize that the three steps of the plan above are com­posed of fur­ther sub-steps. Nested to-do lists cap­ture this well.

Ex­am­ple Re­cur­sive Plan: Host­ing a Din­ner Plan Expanded

  1. In­vite guests

    1. Select date and time

    2. Select guests

    3. Is­sue invitations

  2. Clean house

    1. Call in fa­vor from housemate

  3. Pre­pare food

    1. Buy ingredients

    2. Cook food

Th­ese steps could be fur­ther de­com­posed, but this suffices to demon­strate the idea. We can draw an­other di­a­gram.

Re­cur­sive tree di­a­gram of a plan to host a din­ner party

Why call it re­cur­sion?

Some­times peo­ple use fancy tech­ni­cal terms in or­der to sound smart even though a more com­mon­place word would suffice. Ar­guably, I could have said that plans are hi­er­ar­chi­cal or nested or some­thing. Why use the math­e­mat­ics and com­puter sci­ence term, re­cur­sion?

I defend this choice with refer­ence to the defin­ing fea­tures of re­cur­sion:

  1. You solve a prob­lem by break­ing it into smaller in­stances of the same type.

  2. Each time you de­com­pose the prob­lem into a smaller prob­lem, you ap­ply the same func­tion/​pro­ce­dure to the smaller piece as you did the larger one.

  3. You con­tinue de­com­pos­ing un­til you reach a base case which re­quires no fur­ther de­com­po­si­tion.

  4. The over­all prob­lem is solved by rol­ling up all the solu­tions to lower-level de­com­posed sub-prob­lems.

[The calcu­la­tion of fac­to­ri­als is the canon­i­cal in­tro­duc­tory ex­am­ple for re­cur­sion and neatly illus­trates all of these fea­tures.]

Plan­ning has all of these fea­tures and is there­fore rightly de­scribed as be­ing re­cur­sive.

We see that plans can be de­com­posed into sub-plans which might then be de­com­posed into fur­ther sub-plans. Even­tu­ally you stop, i.e. you reach a base case. When is this? It de­pends on the con­text, but one heuris­tic is to stop when the level is so triv­ial that it doesn’t re­quire fur­ther de­com­po­si­tion. In the above ex­am­ple of host­ing a din­ner party, I might not need to re­curse to the level of plan­ning out the steps of send­ing an emails. I can take for granted that I know how to carry that out.

The Core Plan­ning Process

What is the uni­ver­sal plan­ning func­tion/​pro­ce­dure in the con­text of plan­ning? A de­ci­sion the­o­rist or AI ex­pert might be able to offer a lit­tle more rigor here, but the uni­ver­sal core plan­ning pro­cess we re­peat­edly ap­ply when­ever we are plan­ning is go­ing to look roughly like this:

  • Enu­mer­ate the set of pos­si­ble ac­tions to take.

  • Pre­dict which ac­tions will re­sult in which out­comes (and with which like­li­hood).

  • As­sign rel­a­tive prefer­ences to each of the po­ten­tial out­comes.

  • Use the above points to pri­ori­tize the ac­tions you will take based on the ex­pected costs, benefits, and risks as­so­ci­ated

How the hu­man brain ex­e­cutes the above steps is com­pli­cated. And what is re­quired to ex­e­cute those steps will also differ dra­mat­i­cally be­tween plans and at differ­ent lev­els within a plan. In par­tic­u­lar, sub-plans can re­quire very differ­ent mod­els to map ac­tions to out­comes than the mod­els used at a higher level. The mod­els used for busi­ness strat­egy are differ­ent from those used to file taxes, though both are part of the broader “busi­ness suc­cess” goal. Still, we shouldn’t let the vari­a­tion be­tween plans mask that the core steps are always the same.

Plans are Two-Fold Recursive

We can ac­tu­ally say that plan­ning is dou­bly re­cur­sive. Plan­ning is both re­cur­sive in plan­ning, i.e. when you’re choos­ing what to do [2], and also in ex­e­cu­tion. To ex­e­cute a plan, you must ex­e­cute each of each the sub-plans, and each of the sub-plans’ sub-plans and so on. When we’re talk­ing about ex­e­cu­tion, the re­cur­sive func­tion to be ap­plied at each level is sim­ply “Do X”.

Im­pli­ca­tions of the Re­cur­sive Nature

The fol­low­ing sec­tion de­tails some key prin­ci­ples of good plan­ning that can be di­rectly de­rived from the re­cur­sive na­ture of plans. Even though both the re­cur­sive na­ture and the prin­ci­ples pre­sented here seem sen­si­ble, if not ob­vi­ous, on their own, link­ing the two to­gether makes it clear why these plan­ning prin­ci­ples are nec­es­sar­ily so.

Goals/​Steps/​Ac­tions/​Plans are Isomorphic

When we plan, we usu­ally start with a goal. Then we figure out ac­tions and sub-plans we will take to­wards the goal. This pic­ture is some­what mis­lead­ing. It makes it sound like the goal and steps we take to­wards it are of a differ­ent kind, and so en­courages us to treat them differ­ently. Usu­ally, that means treat­ing the goal as more fixed than it ought to be.

Some­one might de­cide that they have the goal of be­com­ing a lawyer. They start of with this goal and start eval­u­at­ing steps they might take to­wards achiev­ing it. Maybe they start think­ing about which law school they want to go, where they will get a stu­dent loan, how they’re go­ing to im­prove their study habits. They start con­sid­er­ing the ob­sta­cles and how to over­come them.

For each pos­si­ble step to­wards the goal, they enu­mer­ate and eval­u­ate the op­tions for their benefits, costs, and risks. What is so eas­ily missed is whether or not the “goal” was the right step to be­gin with. One must re­mem­ber that the goal it­self is likely only a step within some yet larger step. The goal is a step too be­cause, from the per­spec­tive of the re­cur­sive tree, all the nodes are of the same type. [Per­haps we wish to treat nodes at the very top or very bot­tom differ­ently, but oth­er­wise, there’s no differ­ence.]

It is only in our minds that we dis­t­in­guish be­tween goal, step, ac­tion, and plan. Struc­turally they are the same thing.

See? They’re all just plans.

The ex­cep­tion to the iso­mor­phism are ter­mi­nal goals which we wish to ac­com­plish purely for their own sake rather than in­stru­men­tal goals which we seek to ac­com­plish to at­tain some­thing else. Ter­mi­nal goals are not steps within an­other plan and so the iso­mor­phism does not hold with them. Still, I claim that since the over­whelming ma­jor­ity of our goals and plans are only in­stru­men­tal, it is rea­son­able to as­sert that goals (gener­i­cally) are iso­mor­phic to plans. Most nodes in the re­cur­sive tree are in the mid­dle. It’s true enough that it’s helpful to re­mem­ber it as such.

Al­most All Plans are Sub-Plans; Al­most All Goals are Sub-Goals

The corol­lary to the above point that goals/​steps/​ac­tions/​plans are all the same is that [al­most] all plans are sub-plans and [al­most] all goals are sub-goals. If you have a goal, it’s prob­a­bly be­cause there’s some­thing you hope that goal will ac­com­plish.

We can reuse the lawyer ex­am­ple from above.

What can be lost when you start with your goal and don’t look any higher is all the rea­sons you se­lected your goal in the first place. It is worth re­mem­ber­ing and ex­am­in­ing those rea­sons. Pos­si­bly you se­lected those rea­sons long ago and they no longer ap­ply. Pos­si­bly it was an in­tu­itive or emo­tional de­ci­sion but not an es­pe­cially good one. If you don’t think about the rea­sons be­hind your goals, i.e., your higher-level goals, then you very likely will ei­ther se­lect the wrong goals or se­lect the right ap­prox­i­mate goal which you then ex­e­cute in such a way as to not ac­com­plish your higher-level goals.

Peter Thiel, fa­mous billion­aire in­vestor, went to Stan­ford law school and prac­ticed in a pres­ti­gious New York law firm for seven months and three days. He is oft quoted as de­scribing that this law firm was a place where “from the out­side ev­ery­one wanted to get in and on the in­side ev­ery­body wanted to get out­side.” When he left, a col­league told Thiel that he didn’t know it was “pos­si­ble to leave Al­ca­traz.” Clearly not some­where he ac­tu­ally wanted to be. Peter Thiel cites his rea­son for go­ing to law school in the first place: “So I think I got a JD be­cause of the ab­stract pres­tige, be­cause I didn’t know what else to do, and I didn’t con­cretely think through what it would be like to be a lawyer.”

Peter Thiel’s ex­am­ple high­lights an im­por­tant point. If you know why you have a par­tic­u­lar goal, you can list out the as­sump­tions and be­liefs that cause you to have it. Once listed, the as­sump­tions and be­liefs can be ex­am­ined, fur­ther ev­i­dence can be col­lected, and you can avoid mak­ing plans based on shoddy be­liefs and limited in­for­ma­tion. You can en­sure that you think through con­cretely what it would be like to be a lawyer, or as Thiel ad­vises oth­ers, go iden­tify some­one with a JD who’s a good role model.

The elu­ci­da­tion and ex­am­i­na­tion of as­sump­tions should be part of your “core plan­ning pro­cess” which should car­ried out ap­pro­pri­ately at each level and node of your plan.

In a sense, you need to dis­solve the dis­tinc­tion be­tween plan/​goal/​ac­tion/​step and see the re­cur­sive tree of your plan for what it is. Then you ap­ply your plan­ning pro­cess to the nodes in­dis­crim­i­nately. When start­ing with a goal, you ought to “re­curse up­wards” un­til you iden­tify higher-level goals that you are suffi­ciently con­fi­dent in [3]. Even if you are com­pletely cer­tain about your higher-level goals, you still want to be think­ing about them be­cause their de­tails dic­tate what the de­tails of your lower-level goals need to be.

When our lawyer-wannabe sits down to plan how they’re go­ing to be­come a lawyer, they will make worse choices if they for­get what the point of be­com­ing a lawyer was. The point of a lawyer is not to be­come a lawyer in­her­ently, more likely it’s to have a good job with a high salary, pres­tige, and satis­fy­ing work as part of over­all hav­ing a good life. You can avoid re­peat­ing Peter Thiel’s mis­take.

  1. If you re­mem­ber why you care about a goal, you can ex­am­ine whether this goal will ac­tu­ally achieve your higher level goals. The lawyer-wannabe can look up lawyer job open­ing and salaries, find lawyers she can ques­tion about their lifestyle, poll some friends to see if they ac­tu­ally re­spect lawyers, and so on. She might find out be­ing a lawyer doesn’t serve her higher-level goals and should maybe pur­sue some­thing else. It’d save her a lot of time to find that out sooner rather than later.

  2. It’s pos­si­ble that there are ver­sions of be­com­ing a lawyer which meet the higher level goals and ver­sions which don’t. Be­com­ing a bar­rister might have a lot of pres­tige and high salary yet be stress­ful and have ex­tremely long hours. Con­versely, be­ing an en­vi­ron­men­tal lawyer might pay less but an over­all bet­ter lifestyle. Keep­ing the true goal in mind helps de­ter­mine the de­tails you want to tar­get.

I fear that the very par­tic­u­lar ob­vi­ous ca­reer ex­am­ple I have pre­sented won’t trans­late read­ily in the minds of read­ers to all the other cases where this ap­plies. It ap­plies to pro­jects within star­tups and busi­ness, it ap­plies to the books you read, it ap­plies to the peo­ple you date and how you find them, it ap­plies to your hob­bies and your va­ca­tion. It ap­plies to any plan­ning of non-triv­ial com­plex­ity. I en­courage the reader to search for a few ex­am­ples from their own life where they might benefit from ap­ply­ing this line of think­ing.

A goal is only right if it ac­tu­ally serves your higher-level goals

If you didn’t pick the right goal to be­gin with, then no mat­ter how hard you hit that goal, all is for naught. In terms of a re­cur­sive trees, for a goal to mat­ter, there needs to be an “un­bro­ken path” to some­thing you ter­mi­nally value.

If the lawyer-wannabe suc­ceeds su­per hard at be­com­ing a lawyer: grad­u­ates Summa Cum Laude from the best school, gets a role in the most pres­ti­gious firm, etc., it doesn’t mat­ter at all if it turns out be­ing a lawyer was the wrong choice.

To re­word points from above, goals can be wrong in two ways:

  1. You picked the wrong goal en­tirely. Another op­tion was en­tirely bet­ter.

  2. You picked roughly the right goal, but you im­ple­mented it such that it had no value.

An aside about long-term plan­ning and feedback

High-level goal se­lec­tion is of­ten fraught be­cause there is slow feed­back. A young per­son might not dis­cover for years that they didn’t ac­tu­ally want to be a lawyer, so it’s easy to pick the wrong high-level goal and grind to­wards for years, only get­ting feed­back that it was the wrong choice af­ter they grad­u­ate and start work­ing. Peter Thiel is an ex­am­ple. Another class of ex­am­ples is star­tups and busi­ness de­cid­ing which prod­ucts to build. Done wrong, a startup can pick a long-term goal and work for months or years be­fore find­ing out it was the wrong goal.

Worse than the pure ab­sence of quick-feed­back, long-term plan­ning is of­ten in­fluenced by short-term fac­tors such as emo­tions. “Gah, plan­ning is stress­ful! I’m just go­ing to pick some­thing and stick with it.” Rather than fo­cus on the long-term goal, how the per­son feels in the mo­ment is pri­ori­tized. Re­lat­edly, as with Peter Thiel, in­stead of try­ing to pick the best long-term plan, some­one might pick the plan which gets the great­est so­cial re­ward (since there are in­deed re­wards for just hav­ing cool plans). There can faster feed­back on whether your plan is cool then whether it will suc­ceed. You’ll know as soon as you boast to your friends or de­scribe your product plans to your startup’s in­vestors. The quick feed­back on emo­tions and so­cial re­wards of­ten out­weighs the value of a long-term goal one is sup­pos­edly plan­ning to­wards. Be­ware.

The pre­cise suc­cess crite­ria of a lower-level goal are de­ter­mined by the higher-level goals

The im­por­tance of be­ing clear on why a goal has been se­lected is never clearer than in the case of del­e­ga­tion. A man­ager asks a re­search en­g­ineer to write a re­port on their re­cent re­search. The en­g­ineer does so, but the man­ager is not pleased! The re­port is a tech­ni­cal, suit­able only for other en­g­ineers, yet this is not what the man­ager wanted—the man­ager wanted a re­port to give to the busi­ness ex­ec­u­tives.

The re­search en­g­ineer de­liv­ered what was re­quested “a re­port on their re­cent re­search”, but since this didn’t con­nect with the higher level goal of hav­ing a way to up­date the ex­ec­u­tives, it was worth­less.

The dan­ger is that we hu­mans of­ten en­cap­su­late our goals and plans very crudely and with very lit­tle de­tail. A few words like “be­come a lawyer.” We rely on our­selves and oth­ers im­plic­itly un­der­stand­ing the spe­cific ver­sions of those goals which would count as suc­cess—yet so of­ten we don’t.

Even when the goal is cer­tain, you need to un­der­stand the big­ger pic­ture in or­der to be able to make lower-level im­ple­men­ta­tion choices. Any­one del­e­gat­ing a task is ad­vised to in­struct the del­e­ga­tee in why the task is be­ing re­quested. The more de­tail the bet­ter.

You can’t just ask some­one to book you a venue for an event. You ei­ther need to tell the ex­act kind of venue you want or tell them what kind of event it is so they know what kind is ap­pro­pri­ate. And even if you say “fits 50 peo­ple, within X bud­get, within Y miles of lo­ca­tion A” there could still be fur­ther de­ci­sions whose ideal se­lec­tion is made only if you know why a venue is needed: what kind of event, which guests and there­fore which am­bi­ance and so on would best.

Del­e­ga­tion is hard be­cause it means split­ting the re­cur­sive tree among mul­ti­ple peo­ple. In the case of prin­ci­pal-agent prob­lems, you’re trans­plant­ing a goal from one per­son’s tree to an­other. You can see why that’d run into trou­ble.

Fi­nal ex­am­ple: as a Data Scien­tist, I was of­ten asked to build things with lit­tle speci­fi­ca­tion and no ex­pla­na­tion of why. A tech­ni­cal ex­am­ple here is that if you don’t un­der­stand how a clas­sifier will be used, then even if you can build one, you don’t know how to make a dozen rele­vant de­sign choices, e.g. set­ting thresh­olds which de­ter­mine false pos­i­tives and false nega­tives. Build­ing some­thing with­out an idea of why is a great recipe for build­ing a ver­sion of it which doesn’t meet the true user re­quire­ments. [4]

Wrap­ping up

Though the ideas pre­sented here are sim­ple, I hope that if you have read this far, this post will have deep­ened your ap­pre­ci­a­tion of this fun­da­men­tal prop­erty of plans and will re­sult in you mak­ing bet­ter plans to­wards good things.

Ap­pendix: Mul­ti­ple Goals

Though it was skipped for the sake of sim­plic­ity in the main text, re­al­is­ti­cally, many our plans are ex­e­cuted with more than one goal in mind. This is en­tirely com­pat­i­ble with a re­cur­sive model of plans, but it makes the di­a­grams messier. In or­der to not dis­tract from the main points, ex­am­ples in the text were cho­sen to roll-up neatly into a sin­gle goal.

Below is a re­al­is­tic re­cur­sive plan di­a­gram illus­trat­ing mul­ti­ple goals. Makes you ap­pre­ci­ate what your brain has to deal with.

Modern art: a re­cur­sive tree di­a­gram of a plan where sub-plans feed into mul­ti­ple higher-level plans, e.g. ac­tions are taken to­wards mul­ti­ple goals.

Endnotes

[1] This broad defi­ni­tion stands in op­po­si­tion to a com­mon defi­ni­tion which uses plan­ning pri­mar­ily in con­texts of schedul­ing, e.g. plan your day, plan your week. The broader defi­ni­tion here is es­sen­tially syn­ony­mous with de­ci­sion-mak­ing, per­haps differ­ing only in con­no­ta­tion. De­ci­sions some­what im­ply a one-off choice be­tween op­tions whereas plan­ning im­plies se­lec­tion of mul­ti­ple ac­tions to be taken over time. The term plan­ning also some­what more than de­ci­sion-mak­ing high­lights that there is a goal one wishes to achieve.

[2] With the broad defi­ni­tion of plan­ning used here, it’s im­por­tant to note that plan­ning needn’t hap­pen all at once. More com­monly, differ­ent com­po­nents of a plan (se­lec­tion of ac­tion) hap­pen at differ­ent times. Very com­monly, higher-lev­els of the plans are se­lected up front and lower-level ac­tions are se­lected near or at the time of ex­e­cu­tion.

[3] This is of­ten re­ally, very hard and time-con­sum­ing. If you’re haven’t re­cursed ex­plic­itly up­wards to de­ter­mine what you want and what your true high-level goals are, this might take a while in­clud­ing both the time to do it and pos­si­bly the time spent gain­ing the skills do it. The good news is that af­ter ini­tial in­vest­ment you get to reuse your knowl­edge of your high-level goals re­peat­edly when mak­ing lower-level plans such that you make much bet­ter low-level plans.

[4] My boast: de­ter­mined to not build the wrong thing, I spent enough time figur­ing out what the why rea­sons for our en­g­ineer­ing work should be that my com­pany even­tu­ally asked me to take that over as my ex­plicit role.