Real World Solutions to Prisoners’ Dilemmas

Why should there be real world solu­tions to Pri­son­ers’ Dilem­mas? Be­cause such dilem­mas are a real-world prob­lem.

If I am as­signed to work on a school pro­ject with a group, I can ei­ther co­op­er­ate (work hard on the pro­ject) or defect (slack off while reap­ing the re­wards of ev­ery­one else’s hard work). If ev­ery­one defects, the pro­ject doesn’t get done and we all fail—a bad out­come for ev­ery­one. If I defect but you co­op­er­ate, then I get to spend all day on the beach and still get a good grade—the best out­come for me, the worst for you. And if we all co­op­er­ate, then it’s long hours in the library but at least we pass the class—a “good enough” out­come, though not quite as good as me defect­ing against ev­ery­one else’s co­op­er­a­tion. This ex­actly mir­rors the Pri­soner’s Dilemma.

Di­plo­macy—both the con­cept and the board game—in­volves Pri­son­ers’ Dilem­mas. Sup­pose Ribben­trop of Ger­many and Molo­tov of Rus­sia agree to a peace treaty that de­mil­i­ta­rizes their mu­tual bor­der. If both co­op­er­ate, they can move their forces to other the­aters, and have mod­er­ate suc­cess there—a good enough out­come. If Rus­sia co­op­er­ates but Ger­many defects, it can launch a sur­prise at­tack on an un­defended Rus­sian bor­der and en­joy spec­tac­u­lar suc­cess there (for a while, at least!) - the best out­come for Ger­many and the worst for Rus­sia. But if both defect, then nei­ther has any ad­van­tage at the Ger­man-Rus­sian bor­der, and they lose the use of those troops in other the­aters as well—a bad out­come for both. Again, the Pri­soner’s Dilemma.

Civ­i­liza­tion—again, both the con­cept and the game—in­volves Pri­son­ers’ Dilem­mas. If ev­ery­one fol­lows the rules and cre­ates a sta­ble so­ciety (co­op­er­ates), we all do pretty well. If ev­ery­one else works hard and I turn bar­bar­ian and pillage you (defect), then I get all of your stuff with­out hav­ing to work for it and you get noth­ing—the best solu­tion for me, the worst for you. If ev­ery­one be­comes a bar­bar­ian, there’s noth­ing to steal and we all lose out. Pri­soner’s Dilemma.

If ev­ery­one who wor­ries about global warm­ing co­op­er­ates in cut­ting emis­sions, cli­mate change is averted and ev­ery­one is mod­er­ately happy. If ev­ery­one else co­op­er­ates in cut­ting emis­sions, but one coun­try defects, cli­mate change is still mostly averted, and the defec­tor is at a sig­nifi­cant eco­nomic ad­van­tage. If ev­ery­one defects and keeps pol­lut­ing, the cli­mate changes and ev­ery­one loses out. Again a Pri­soner’s Dilemma,

Pri­son­ers’ Dilem­mas even come up in na­ture. In ba­boon tribes, when a fe­male is in “heat”, males of­ten com­pete for the chance to woo her. The most suc­cess­ful males are those who can get a friend to help fight off the other mon­keys, and who then helps that friend find his own mon­key lov­ing. But these mon­keys are tempted to take their friend’s fe­male as well. Two males who co­op­er­ate each se­duce one fe­male. If one co­op­er­ates and the other defects, he has a good chance at both fe­males. But if the two can’t co­op­er­ate at all, then they will be beaten off by other mon­key al­li­ances and won’t get to have sex with any­one. Still a Pri­soner’s Dilemma!

So one might ex­pect the real world to have pro­duced some prac­ti­cal solu­tions to Pri­son­ers’ Dilem­mas.

One of the best known such sys­tems is called “so­ciety”. You may have heard of it. It boasts a se­ries of norms, laws, and au­thor­ity figures who will pun­ish you when those norms and laws are bro­ken.

Imag­ine that the two crim­i­nals in the origi­nal ex­am­ple were part of a crim­i­nal so­ciety—let’s say the Mafia. The God­father makes Alice and Bob an offer they can’t re­fuse: turn against one an­other, and they will end up “sleep­ing with the fishes” (this con­cludes my knowl­edge of the Mafia). Now the in­cen­tives are changed: defect­ing against a co­op­er­a­tor doesn’t mean walk­ing free, it means get­ting mur­dered.

Both pris­on­ers co­op­er­ate, and amaz­ingly the threat of mur­der ends up mak­ing them both bet­ter off (this is also the gist of some of the strongest ar­gu­ments against liber­tar­i­anism: in Pri­soner’s Dilem­mas, threat­en­ing force against ra­tio­nal agents can in­crease the util­ity of all of them!)

Even when there is no god­father, so­ciety binds peo­ple by con­cern about their “rep­u­ta­tion”. If Bob got a rep­u­ta­tion as a snitch, he might never be able to work as a crim­i­nal again. If a stu­dent gets a rep­u­ta­tion for slack­ing off on pro­jects, she might get os­tra­cized on the play­ground. If a coun­try gets a rep­u­ta­tion for back­stab­bing, oth­ers might re­fuse to make treaties with them. If a per­son gets a rep­u­ta­tion as a ban­dit, she might in­cur the hos­tility of those around her. If a coun­try gets a rep­u­ta­tion for not do­ing enough to fight global warm­ing, it might...well, no one ever said it was a perfect sys­tem.

Aside from hu­mans in so­ciety, evolu­tion is also strongly mo­ti­vated to de­velop a solu­tion to the Pri­soner’s Dilemma. The Dilemma trou­bles not only lovestruck ba­boons, but ants, min­nows, bats, and even viruses. Here the pay­off is de­nom­i­nated not in years of jail time, nor in dol­lars, but in re­pro­duc­tive fit­ness and num­ber of po­ten­tial offspring—so evolu­tion will cer­tainly take note.

Most peo­ple, when they hear the ra­tio­nal ar­gu­ments in fa­vor of defect­ing ev­ery sin­gle time on the iter­ated 100-crime Pri­soner’s Dilemma, will feel some kind of emo­tional re­sis­tance. Thoughts like “Well, maybe I’ll try co­op­er­at­ing any­way a few times, see if it works”, or “If I promised to co­op­er­ate with my op­po­nent, then it would be dishon­or­able for me to defect on the last turn, even if it helps me out., or even “Bob is my friend! Think of all the good times we’ve had to­gether, rob­bing banks and run­ning straight into wait­ing po­lice cor­dons. I could never be­tray him!”

And if two peo­ple with these sorts of emo­tional hangups play the Pri­soner’s Dilemma to­gether, they’ll end up co­op­er­at­ing on all hun­dred crimes, get­ting out of jail in a mere cen­tury and leav­ing ra­tio­nal util­ity max­i­miz­ers to sit back and won­der how they did it.

Here’s how: imag­ine you are a su­pervillain de­sign­ing a robotic crim­i­nal (who’s that go-to su­pervillain Kaj always uses for situ­a­tions like this? Dr. Zany? Okay, let’s say you’re him). You ex­pect to build sev­eral copies of this robot to work as a team, and ex­pect they might end up play­ing the Pri­soner’s Dilemma against each other. You want them out of jail as fast as pos­si­ble so they can get back to fur­ther­ing your ne­far­i­ous plots. So rather than have them bum­ble through the whole ra­tio­nal util­ity max­i­miz­ing thing, you just in­sert an ex­tra line of code: “in a Pri­soner’s Dilemma, always co­op­er­ate with other robots”. Prob­lem solved.

Evolu­tion fol­lowed the same strat­egy (no it didn’t; this is a mas­sive over­sim­plifi­ca­tion). The emo­tions we feel around friend­ship, trust, al­tru­ism, and be­trayal are partly a built-in hack to suc­ceed in co­op­er­at­ing on Pri­soner’s Dilem­mas where a ra­tio­nal util­ity-max­i­mizer would defect a hun­dred times and fail mis­er­ably. The evolu­tion­ar­ily dom­i­nant strat­egy is com­monly called “Tit-for-tat”—ba­si­cally, co­op­er­ate if and only if your op­po­nent did so last time.

This so-called “su­per­ra­tional­ity” ap­pears even more clearly in the Ul­ti­ma­tum Game. Two play­ers are given $100 to dis­tribute among them­selves in the fol­low­ing way: the first player pro­poses a dis­tri­bu­tion (for ex­am­ple, “Fifty for me, fifty for you”) and then the sec­ond player ei­ther ac­cepts or re­jects the dis­tri­bu­tion. If the sec­ond player ac­cepts, the play­ers get the money in that par­tic­u­lar ra­tio. If the sec­ond player re­fuses, no one gets any money at all.

The first player’s rea­son­ing goes like this: “If I pro­pose $99 for my­self and $1 for my op­po­nent, that means I get a lot of money and my op­po­nent still has to ac­cept. After all, she prefers $1 to $0, which is what she’ll get if she re­fuses.

In the Pri­soner’s Dilemma, when play­ers were able to com­mu­ni­cate be­fore­hand they could set­tle upon a win­ning strat­egy of pre­com­mit­ing to re­cip­ro­cate: to take an ac­tion benefi­cial to their op­po­nent if and only if their op­po­nent took an ac­tion benefi­cial to them. Here, the sec­ond player should con­sider the same strat­egy: pre­com­mit to an ul­ti­ma­tum (hence the name) that un­less Player 1 dis­tributes the money 50-50, she will re­ject the offer.

But as in the Pri­soner’s Dilemma, this fails when you have no rea­son to ex­pect your op­po­nent to fol­low through on her pre­com­mit­ment. Imag­ine you’re Player 2, play­ing a sin­gle Ul­ti­ma­tum Game against an op­po­nent you never ex­pect to meet again. You du­tifully promise Player 1 that you will re­ject any offer less than 50-50. Player 1 offers 80-20 any­way. You rea­son “Well, my ul­ti­ma­tum failed. If I stick to it any­way, I walk away with noth­ing. I might as well ad­mit it was a good try, give in, and take the $20. After all, re­ject­ing the offer won’t mag­i­cally bring my chance at $50 back, and there aren’t any other deal­ings with this Player 1 guy for it to in­fluence.”

This is seem­ingly a ra­tio­nal way to think, but if Player 1 knows you’re go­ing to think that way, she offers 99-1, same as be­fore, no mat­ter how sincere your ul­ti­ma­tum sounds.

No­tice all the similar­i­ties to the Pri­soner’s Dilemma: play­ing as a “ra­tio­nal eco­nomic agent” gets you a bad re­sult, it looks like you can es­cape that bad re­sult by mak­ing pre­com­mit­ments, but since the other player can’t trust your pre­com­mit­ments, you’re right back where you started

If evolu­tion­ary solu­tions to the Pri­son­ers’ Dilemma look like trust or friend­ship or al­tru­ism, solu­tions to the Ul­ti­ma­tum Game in­volve differ­ent emo­tions en­tirely. The Sul­tan pre­sum­ably does not want you to elope with his daugh­ter. He makes an ul­ti­ma­tum: “Touch my daugh­ter, and I will kill you.” You elope with her any­way, and when his guards drag you back to his palace, you ar­gue: “Killing me isn’t go­ing to re­verse what hap­pened. Your ul­ti­ma­tum has failed. All you can do now by be­head­ing me is get blood all over your beau­tiful palace car­pet, which hurts you as well as me—the equiv­a­lent of pointlessly pass­ing up the last dol­lar in an Ul­ti­ma­tum Game where you’ve just been offered a 99-1 split.”

The Sul­tan might counter with an ar­gu­ment from so­cial in­sti­tu­tions: “If I let you go, I will look dishon­or­able. I will gain a rep­u­ta­tion as some­one peo­ple can mess with with­out any con­se­quences. My choice isn’t be­tween bloody car­pet and clean car­pet, it’s be­tween bloody car­pet and peo­ple re­spect­ing my or­ders, or clean car­pet and peo­ple con­tin­u­ing to defy me.”

But he’s much more likely to just shout an in­co­her­ent stream of dread­ful Ara­bic curse words. Be­cause just as friend­ship is the evolu­tion­ary solu­tion to a Pri­soner’s Dilemma, so anger is the evolu­tion­ary solu­tion to an Ul­ti­ma­tum Game. As var­i­ous gu­rus and psy­chol­o­gists have ob­served, anger makes us ir­ra­tional. But this is the good kind of ir­ra­tional­ity; it’s the kind of ir­ra­tional­ity that makes us pass up a 99-1 split even though the de­ci­sion costs us a dol­lar.

And if we know that hu­mans are the kind of life-form that tends to ex­pe­rience anger, then if we’re play­ing an Ul­ti­ma­tum Game against a hu­man, and that hu­man pre­com­mits to re­ject­ing any offer less than 50-50, we’re much more likely to be­lieve her than if we were play­ing against a ra­tio­nal util­ity-max­i­miz­ing agent—and so much more likely to give the hu­man a fair offer.

It is dis­taste­ful and a lit­tle bit con­tra­dic­tory to the spirit of ra­tio­nal­ity to be­lieve it should lose out so badly to sim­ple emo­tion, and the prob­lem might be cor­rectable. Here we risk cross­ing the poorly charted bor­der be­tween game the­ory and de­ci­sion the­ory and reach­ing ideas like time­less de­ci­sion the­ory: that one should act as if one’s choices de­ter­mined the out­put of the al­gorithm one in­stan­ti­ates (or more sim­ply, you should as­sume ev­ery­one like you will make the same choice you do, and take that into ac­count when choos­ing.)

More prac­ti­cally, how­ever, most real-world solu­tions to Pri­soner’s Dilem­mas and Ul­ti­ma­tum Games still hinge on one of three things: threats of re­cip­ro­ca­tion when the length of the game is un­known, so­cial in­sti­tu­tions and rep­u­ta­tion sys­tems that make defec­tion less at­trac­tive, and emo­tions rang­ing from co­op­er­a­tion to anger that are hard-wired into us by evolu­tion. In the next post, we’ll look at how these play out in prac­tice.