Compressing Reality to Math

This is part of a se­quence on de­ci­sion anal­y­sis and fol­lows 5 Ax­ioms of De­ci­sion-Mak­ing, which ex­plains how to turn a well-formed prob­lem into a solu­tion. Here we dis­cuss turn­ing re­al­ity into a well-formed prob­lem. There are three ba­sic ac­tions I’d like to in­tro­duce, and then work through some ex­am­ples.

Scope

The first thing you have to de­cide with a prob­lem is, well, what the prob­lem is. Sup­pose you’re con­tem­plat­ing re­mod­el­ing your kitchen, and the con­trac­tor you’re look­ing at offers mar­ble or gran­ite coun­ter­tops. While de­cid­ing whether you want mar­ble or gran­ite, you stop and won­der- is this re­ally the con­trac­tor that you should be us­ing? Ac­tu­ally, should you even be re­mod­el­ing your kitchen? Maybe you should to move to a bet­ter city first. But if you’re already think­ing about mov­ing, you might even want to em­i­grate to an­other coun­try.

At this point the con­trac­tor awk­wardly coughs and asks whether you’d like mar­ble or gran­ite.

De­ci­sions take effort to solve, es­pe­cially if you’re try­ing to care­fully avoid bias. It helps to par­ti­tion the world and deal with lo­cal prob­lems- you can figure out which coun­ter­tops you want with­out first figur­ing out what coun­try you want to live in. It’s also im­por­tant to keep in mind lost pur­poses- if you’re go­ing to move to a new city, re­mod­el­ing your kitchen is prob­a­bly a mis­take, even af­ter you already called a con­trac­tor. Be open to go­ing up a level, but not par­a­lyzed by the pos­si­bil­ity, which is a care­ful bal­anc­ing act. Spend­ing time pe­ri­od­i­cally go­ing up lev­els and reeval­u­at­ing your de­ci­sions and di­rec­tions can help, as well as hav­ing a philos­o­phy of life.

Model

Now that you’ve got a first draft of what your prob­lem en­tails, how does that cor­ner of the world work? What are the key de­ci­sions and the key un­cer­tain­ties? A tool that can be of great help here is an in­fluence di­a­gram, which is a di­rected acyclic graph1 which rep­re­sents the un­cer­tain­ties, de­ci­sions, and val­ues in­her­ent in a prob­lem. While sketch­ing out your model, do you be­come more or less com­fortable with the scope of the de­ci­sion? If you’re less com­fortable, move up (or down) a level and re­model.

Elicit

Now that you have an in­fluence di­a­gram, you need to pop­u­late it with num­bers. What (con­di­tional) prob­a­bil­ities do you as­sign to un­cer­tainty nodes? What prefer­ences do you as­sign to pos­si­ble out­comes? Are there any other un­cer­tainty nodes you could add to clar­ify your calcu­la­tions? (For ex­am­ple, when mak­ing a de­ci­sion based on a med­i­cal test, you may want to add a “un­der­ly­ing re­al­ity” node that in­fluences the test re­sults but that you can’t see, to make it eas­ier to elicit prob­a­bil­ities.)

Out­ing with a Friend

A friend calls me up: “Let’s do some­thing this week­end.” I agree, and pon­der my op­tions. We typ­i­cally play Go, and I can ei­ther host at my apart­ment or we could meet at a lo­cal park. If it rains, the park will not be a fun place to be; if it’s sunny, though, the park is much nicer than my apart­ment. I check the weather fore­cast for Satur­day and am not im­pressed: 50% chance of rain.2 I check cal­ibra­tion data and see that for a 3 day fore­cast, a 50% pre­dic­tion is well cal­ibrated, so I’ll just use that num­ber.3

If we play Go, we’re locked in to what­ever lo­ca­tion we choose, be­cause mov­ing the board would be a gi­ant has­sle.4 But we could just talk- it’s been a while since we caught up. I don’t think I would en­joy that as much as play­ing Go, but it would give us lo­ca­tion flex­i­bil­ity- if it’s nice out we’ll go to the park, and if it starts rain­ing we can just walk back to my apart­ment.

Note that I just added a de­ci­sion to the prob­lem, and so I up­date my di­a­gram ac­cord­ingly. With in­fluence di­a­grams, you have a choice of how de­tailed to be- I could have made the ac­tivity de­ci­sion point to two branches, one in which I see the weather then pick my lo­ca­tion, and an­other where I pick my lo­ca­tion and then see the weather. I chose a more stream­lined ver­sion, in which I choose be­tween play­ing Go or walk­ing wher­ever the rain isn’t (where the choice of lo­ca­tion isn’t part of my op­ti­miza­tion prob­lem, since I con­sider it ob­vi­ous.

Com­ing up with cre­ative op­tions like this is one of the ma­jor benefits of care­ful de­ci­sion-mak­ing. When you eval­u­ate ev­ery part of the de­ci­sion and make ex­plicit your de­pen­den­cies, you can see holes in your model or places where you can make your­self bet­ter off by re­cast­ing the prob­lem. Sim­ply get­ting ev­ery­thing on pa­per in a vi­sual for­mat can do won­ders to help clar­ify and solve prob­lems.

At this point, I’m happy with my model- I could come up with other op­tions, but this is prob­a­bly enough for a prob­lem of this size.5 I’ve got six out­comes- ei­ther (we walk, play go (in­side, out­side)) and it (rains, doesn’t rain). I de­cide that I would most pre­fer play­ing go out­side and it doesn’t rain, and would least pre­fer play­ing go out­side and it rains.6 Order­ing the rest is some­what difficult, but I come up with the fol­low­ing ma­trix:

If we walk, I won’t en­joy it quite as much as play­ing Go, but if we play Go in my apart­ment and it’s sunny I’ll re­gret that we didn’t play out­side. Note that all of the prefer­ences prob­a­bil­ities are fairly high- that’s be­cause hav­ing a game in­ter­rupted by rain is much worse than all of the other out­comes. Calcu­lat­ing the prefer­ence value of each deal is easy, since the prob­a­bil­ity of rain is in­de­pen­dent of my de­ci­sion and .5: I de­cide that play­ing Go at Park is the worst op­tion with an effec­tive .5 chance of the best out­come, Go at Home is bet­ter with an effec­tive .7 chance of the best out­come, and Walk is best with an effec­tive .75 chance of the best out­come.

Note that work­ing through ev­ery­thing worked out bet­ter for us than do­ing sce­nario plan­ning. If I knew it would rain, I would choose Go at Home; if I knew it wouldn’t rain, I would choose Go at Park. Walk is dom­i­nated by both of those in the case of cer­tainty, but its lower var­i­ance means it wins out when I’m very un­cer­tain about whether it will rain or not.7

What’s go­ing up a level here? Eval­u­at­ing whether or not I want to do some­thing with this friend this week­end (and an­other level might be eval­u­at­ing whether or not I want them as a friend, and so on). When eval­u­at­ing the prospects, I might want to com­pare them to what­ever I was plan­ning be­fore my friend called to make sure this is ac­tu­ally a bet­ter plan. It could be the chance of rain is so high it makes this plan worse than what­ever al­ter­na­tives I had be­fore I knew it was likely to rain.

Manag­ing a Store

Sup­pose I man­age a store that sells a va­ri­ety of prod­ucts. I de­cide that I want to max­i­mize prof­its over the course of a year, plus some es­ti­mate of in­tan­gibles (like cus­tomer satis­fac­tion). I have a num­ber of year-long com­mit­ments (the lease on my space, em­ploy­ment con­tracts for full-time em­ploy­ees, etc.) already made, which I’ll con­sider be­yond the scope of the prob­lem. I then have a num­ber of de­ci­sions I have to make month-to-month8 (how many sea­sonal em­ploy­ees to hire, what prod­ucts to or­der, whether to change wages or prices, what hours the store should be open), and then de­ci­sions I have to make day-to-day (which em­ploy­ees to sched­ule, where to place items in the store, where to place em­ploy­ees in the store, how to spend my time).

I look at the day-to-day de­ci­sions and de­cide that I’m happy mod­el­ing those as poli­cies rather than in­di­vi­d­ual de­ci­sions- I don’t need to have mapped those out now, but I do need to put my Jan­uary or­ders in next week. Which poli­cies I adopt might be rele­vant to my de­ci­sion, though, and so I still want to model them on that level.

Well, what about my un­cer­tain­ties? Em­ployee morale seems like one that’ll vary month-to-month or day-to-day, though I’m com­fortable mod­el­ing it month-to-month, as that’s when I would change the em­ployee com­po­si­tion or wages. Cus­tomer satis­fac­tion is an un­cer­tainty that seems like it would be worth track­ing, and so is cus­tomer de­mand- how the prices I set will in­fluence sales. And I should model de­mand by item- or maybe just by cat­e­gory of items. La­bor costs, in­ven­tory costs, and rev­enue are all nodes that I could stick in as un­cer­tainty nodes (even though they might just de­ter­minis­ti­cally calcu­late those based on their in­put nodes).

You can imag­ine that the in­fluence di­a­gram I’m sketch­ing is start­ing to get mas­sive- and I’m just con­sid­er­ing this year! I also need to think care­fully about how I put the de­pen­den­cies in this net­work- should I have cus­tomer satis­fac­tion point to cus­tomer de­mand, or the other way around? For un­cer­tainty nodes it doesn’t mat­ter much (as we know how to flip them), but may make elic­i­ta­tion eas­ier or harder. For de­ci­sion nodes, or­der is crit­i­cal, as it rep­re­sents the or­der the de­ci­sions have to be made in.

But even though the prob­lem is get­ting mas­sive, I can still use this method­ol­ogy to solve it. This’ll make it eas­ier for me to keep ev­ery­thing in mind- be­cause I won’t need to keep ev­ery­thing in mind. I can con­tem­plate the de­pen­den­cies and un­cer­tain­ties of the sys­tem one piece at a time, record the re­sults, and then in­te­grate them to­gether. Like with the pre­vi­ous ex­am­ple, it’s easy to in­clude the re­sults of other peo­ple’s calcu­la­tions in your de­ci­sion prob­lem as a node, mean­ing that this can be ex­tended to team de­ci­sions as well- maybe my mar­keter does all the elic­i­ta­tion for the de­mand nodes, and my as­sis­tant man­ager does all the elic­i­ta­tion for the em­ployee nodes, and then I can com­bine them into one de­ci­sion-mak­ing net­work.

New­comb’s Problem

New­comb’s Prob­lem is a thought ex­per­i­ment de­signed to high­light the differ­ence be­tween differ­ent de­ci­sion the­o­ries that has come up a lot on Less Wrong. An­naSala­mon wrote an ar­ti­cle (that even in­cludes in­fluence di­a­grams that are much pret­tier than mine) an­a­lyz­ing New­comb’s Prob­lem, in which she pre­sents three ways to in­ter­pret the prob­lem. I won’t re­peat her anal­y­sis, but will make sev­eral ob­ser­va­tions:

1. Prob­lem state­ments, and real life, are of­ten am­bigu­ous or un­cer­tain. Nav­i­gat­ing that am­bi­guity and un­cer­tainty about what prob­lem you’re ac­tu­ally fac­ing is a ma­jor com­po­nent of mak­ing de­ci­sions. It’s also a ma­jor place for bias to creep in: if you aren’t care­ful about defin­ing your prob­lems, they will be defined in care­less ways, which can im­pose real and large costs in worse solu­tions.

2. It’s easy to con­struct a thought ex­per­i­ment with con­tra­dic­tory premises, and not no­tice if you keep math and pic­tures out of it. Draw pic­tures, show the math. It makes nor­mal prob­lems eas­ier, and helps you no­tice when a prob­lem boils down to “could an un­stop­pable force move an im­mov­able ob­ject?”,9 and then you can move on.

3. If you’re not quite sure which of sev­eral in­ter­pre­ta­tions is true, you can model that ex­plic­itly. Put an un­cer­tainty node at the top which points to the model where your be­hav­ior and Omega’s de­ci­sion are in­de­pen­dent and the model where your be­hav­ior de­ter­mines Omega’s be­hav­ior. Elicit a prob­a­bil­ity, or calcu­late what the prob­a­bil­ity would need to be for you to make one de­ci­sion and then com­pare that to how un­cer­tain you are.

1. This means that there are a bunch of nodes con­nected by ar­rows (di­rected edges), and that there are no cy­cles (ar­rows that point in a loop). As a con­se­quence, there are some nodes with no ar­rows point­ing to them and one node with no ar­rows leav­ing it (the value node). It’s also worth men­tion­ing that in­fluence di­a­grams are a spe­cial case of Bayesian Net­works.

2. This is the ac­tual pre­dic­tion for me as of 1:30 PM on 12/​14/​2011. Link here, though that link should be­come worth­less by to­mor­row. Also note that I’m as­sum­ing that rain will only hap­pen af­ter we start the game of Go /​ de­cide where to play, or that the switch­ing costs will be large.

3. The ac­tual per­centage in the data they had col­lected by 2008 was ~53%, but as it was within the ‘well cal­ibrated’ re­gion I should use the num­ber at face value.

4. If it started rain­ing while we were play­ing out­side, we would prob­a­bly just stop and do some­thing else rather than play­ing through the rain, but nei­ther of those are at­trac­tive prospects.

5. I strongly recom­mend satis­fic­ing, in the AI sense of in­clud­ing the costs of op­ti­miz­ing in your op­ti­miza­tion pro­cess, rather than max­i­miza­tion. Op­por­tu­nity costs are real. For a prob­lem with, say, mil­lions of dol­lars on the line, you’ll prob­a­bly want to spend quite a bit of time try­ing to come up with other op­tions.

6. You may have no­ticed a trend that the best and worst out­come are of­ten paired to­gether. This is more likely in con­structed prob­lems, but is a fea­ture of many difficult real-world prob­lems.

7. We can do some­thing called sen­si­tivity anal­y­sis to see how sen­si­tive this re­sult is; if it’s very sen­si­tive to a value we elic­ited, we might go back and see if we can nar­row the un­cer­tainty on that value. If it’s not very sen­si­tive, then we don’t need to worry much about those un­cer­tain­ties.

8. This is an ar­tifi­cial con­straint- re­ally, you could change any of those poli­cies any day, or even at any time dur­ing that day. It’s of­ten helpful to chunk con­tin­u­ous pe­ri­ods into a few dis­crete ones, but only to the de­gree that your bins carve re­al­ity at the joints.

9. The un­stop­pable force be­ing Omega’s pre­dic­tion abil­ity, and the im­mov­able ob­ject be­ing causal­ity only prop­a­gat­ing for­ward in time. Two-box­ers an­swer “un­mov­able ob­ject,” one-box­ers an­swer “un­stop­pable force.”

• It’s easy to con­struct a thought ex­per­i­ment with con­tra­dic­tory premises, and not no­tice if you keep math and pic­tures out of it. Draw pic­tures, show the math. It makes nor­mal prob­lems eas­ier, and helps you no­tice when a prob­lem boils down to “could an un­stop­pable force move an im­mov­able ob­ject?”, and then you can move on.

The un­stop­pable force be­ing Omega’s pre­dic­tion abil­ity, and the im­mov­able ob­ject be­ing causal­ity only prop­a­gat­ing for­ward in time. Two-box­ers an­swer “un­mov­able ob­ject,” one-box­ers an­swer “un­stop­pable force.”

I’m con­fused by what you mean here about New­comb’s prob­lem, if you say that two-box­ers and one-box­ers are both failing to dis­solve the ques­tion. Do you mean that one par­tic­u­lar ar­gu­ment for two-box­ing and one par­tic­u­lar ar­gu­ment for one-box­ing are failures to dis­solve the ques­tion (but that one or the other de­ci­sion is still valid)? Or do you mean that New­comb’s Prob­lem is an im­pos­si­ble situ­a­tion which doesn’t ap­prox­i­mate any­thing we might re­ally face, and there­fore we should stop wor­ry­ing about it and move on? Or do you mean some­thing differ­ent en­tirely?

• Or do you mean that New­comb’s Prob­lem is an im­pos­si­ble situ­a­tion which doesn’t ap­prox­i­mate any­thing we might re­ally face, and there­fore we should stop wor­ry­ing about it and move on?

This is my per­sonal opinion, though I do not ex­pect ev­ery­one to agree. I would go fur­ther and say that its con­nec­tion to real situ­a­tions is too ten­u­ous to be valuable- many im­pos­si­ble sce­nar­ios are worth think­ing about be­cause they closely ap­prox­i­mate real sce­nar­ios. The pre­vi­ous sen­tence- that any one ex­pla­na­tion of why two-box­ing or one-box­ing is cor­rect fails to un­der­stand why it’s a prob­lem- is some­thing I ex­pect ev­ery­one should agree with.

• Well, then we have a big­ger dis­agree­ment at stake. Also:

The pre­vi­ous sen­tence- that any one ex­pla­na­tion of why two-box­ing or one-box­ing is cor­rect fails to un­der­stand why it’s a prob­lem- is some­thing I ex­pect ev­ery­one should agree with.

You switched the quan­tifier on my ques­tion, and your ver­sion is much stronger than any­thing I’d agree to, much less ex­pect to be un­con­tro­ver­sial.

• Good post, up­voted for giv­ing prac­ti­cal ad­vice backed by deep the­o­ries. It did raise a few ques­tions, but on re­flec­tion most are more meta-is­sues about LW that show up in this se­quence than about the post it­self. So, I think I’ll kick them over to their own dis­cus­sion post. That said I am cu­ri­ous if you know of any large scale stud­ies about the re­sults of peo­ple un­der­go­ing train­ing in us­ing de­ci­sion the­ory. -edit dis­cu­sion post is here

• That said I am cu­ri­ous if you know of any large scale stud­ies about the re­sults of peo­ple un­der­go­ing train­ing in us­ing de­ci­sion the­ory.

What sort of met­ric would you want the study to mea­sure? I know Chevron has been work­ing on in­te­grat­ing this style of DA into their cor­po­rate cul­ture for 20 years, and they seem to think it works. I’m less cer­tain of how valuable it is, or what kinds of value it im­proves, for a per­son on the street, but am gen­er­ally op­ti­mistic about these sorts of clear-think­ing pro­ce­dures.

• For busi­ness or di­vi­sions within the same busi­ness the mea­sures that are already used to mea­sure suc­cess, es­pe­cially those that are close to their ter­mi­nal goals would likely be suffi­cient. It would be pretty hard to come up with a good mea­sure for in­di­vi­d­u­als. I would say their level of im­prove­ment on goals they set for them­selves, but un­der­stand how that would be hard to use as the ba­sis for a study. Per­haps hap­piness or money as those are both very com­mon ma­jor goals.

• Voted up for the maths and clear ex­po­si­tion.