The Scylla of Error and the Charybdis of Paralysis

We’re in­ter­ested in im­prov­ing hu­man ra­tio­nal­ity. Many of our tech­niques for im­prov­ing hu­man ra­tio­nal­ity take time. In real-time situ­a­tions, you can lose by mak­ing the wrong de­ci­sion, or by mak­ing the “right” de­ci­sion too slowly. Most of us do not have in­flex­ible-sched­ule, high-stakes de­ci­sions to make, though. How of­ten does real-time de­ci­sion mak­ing re­ally come up?

Sup­pose you are mak­ing a fairly long-ranged de­ci­sion. Call this de­ci­sion 1. While an­a­lyz­ing de­ci­sion 1, you come to a nat­u­ral pause. At this pause you need to de­cide whether to an­a­lyze fur­ther, or to act on your best-so-far anal­y­sis. Call this de­ci­sion 2. Note that de­ci­sion 2 is made un­der tighter time pres­sure than de­ci­sion 1. This sce­nario ar­gues that de­ci­sion-mak­ing is re­cur­sive, and so if there are any time bounds, then many de­ci­sions will need to be made at very tight time bounds.

A sec­ond, “covert” goal of this post is to provide a definitely-not-para­dox­i­cal prob­lem for peo­ple to prac­tice their Bay­seian rea­son­ing on. Here is a con­crete model of real-time de­ci­sion­mak­ing, mo­ti­vated by med­i­cal-drama tele­vi­sion shows, where the team di­ag­noses and treats a pa­tient over the course of each epi­sode. Di­ag­nos­ing and treat­ing a pa­tient who is dy­ing of an un­known dis­ease is a col­or­ful ex­am­ple of real-time de­ci­sion­mak­ing.

To play this game, you need a coin, two six-sided dice, a deck of cards, and a helper to ma­nipu­late these ob­jects. The ma­nipu­la­tor sets up the game by flip­ping a coin. If heads (tails) the pa­tient is suffer­ing from an ex­otic fun­gus (allergy). Then the ma­nipu­la­tor pre­pares a deck by re­mov­ing all of the clubs (di­a­monds) so that the deck is a red-bi­ased (black-bi­ased) ran­dom-color gen­er­a­tor. Fi­nally, the ma­nipu­la­tor de­ter­mines the pa­tients start­ing health by rol­ling the dice and sum­ming them. All of this is done se­cretly.

Play pro­ceeds in turns. At the be­gin­ning of each turn, the ma­nipu­la­tor flips a coin to de­ter­mine whether test re­sults are available. If test re­sults are available, the ma­nipu­la­tor draws a card from the deck and re­ports its color. A red (black) card gives you sug­ges­tive ev­i­dence that the pa­tient is suffer­ing from a fun­gus (allergy). You choose whether to treat a fun­gus, allergy, or wait. If you treat cor­rectly, the ma­nipu­la­tor leaves the pa­tient’s health where it is (they’re im­prov­ing, but on a longer timescale). If you wait, the ma­nipu­la­tor re­duces the pa­tient’s health by one. If you treat in­cor­rectly, the ma­nipu­la­tor re­duces the pa­tient’s health by two.

Play ends when you treat the pa­tient for the same dis­ease for six con­sec­u­tive turns or when the pa­tient reaches zero health.

Here is some Python code simu­lat­ing a sim­plis­tic strat­egy. What Bayesian strat­egy yields the best re­sults? Is there a con­cise de­scrip­tion of this strat­egy?

The model can be made more com­pli­cated. The space of pos­si­ble ac­tions is small. There is no choice of what to in­ves­ti­gate next. In the real world, there are likely to be diminish­ing re­turns to fur­ther tests or fur­ther anal­y­sis. There could be un­cer­tainty about how much time pres­sure there is. There could be un­cer­tainty about how much in­for­ma­tion fu­ture tests will re­veal. Every com­pli­ca­tion will make the task of com­put­ing the best strat­egy more difficult.

We need fast ap­prox­i­ma­tions to ra­tio­nal­ity (even quite bad ap­prox­i­ma­tions, if they’re fast enough), as well as pro­ce­dures that spend time in or­der to pur­chase a bet­ter re­sult.