Diplomacy as a Game Theory Laboratory

Game the­ory. You’ve stud­ied the posts, you’ve laughed at the comics, you’ve heard the mu­sic1. But the best way to make it Truly Part Of You is to play a gen­uine game, and I have yet to find any more effec­tive than Di­plo­macy.

Di­plo­macy is a board game for seven peo­ple played on a map of WWI Europe. The goal is to cap­ture as many strate­gic provinces (“sup­ply cen­ters”) as pos­si­ble; eigh­teen are needed to win. But each player’s coun­try starts off with the same sized army, and there is no luck or op­por­tu­nity for es­pe­cially clever tac­tics. The most com­mon way to defeat an en­emy is to form coal­i­tions with other play­ers. But your en­e­mies will also be try­ing to form coal­i­tions, and the most prof­itable move is of­ten to be a “dou­ble agent”, string­ing both coun­tries along as long as you can. All game moves are writ­ten in se­cret and re­vealed at the same time and there are no en­force­ment mechanisms, so al­li­ances, de­spite their cen­tral im­por­tance, aren’t always worth the pa­per they’re printed on.

The con­di­tions of Di­plo­macy—com­pe­ti­tion for scarce re­sources, ra­tio­nal self-in­ter­ested ac­tors, im­por­tance of coal­i­tions, lack of ex­ter­nal en­force­ment mechanisms—mir­ror the con­di­tions of game the­o­retic situ­a­tions like the Pri­soner’s Dilemma (and the con­di­tions of most of hu­man evolu­tion!) and so make a sur­pris­ingly pow­er­ful lab­o­ra­tory for an­a­lyz­ing con­cepts like trust, friend­ship, gov­ern­ment, and even re­li­gion.

Over the past few months, I’ve played two on­line games of Di­plo­macy. One I won through a par­tic­u­larly in­ter­est­ing method; the other I lost quite badly, but with an un­usual con­so­la­tion. This post is based on notes I took dur­ing the games about rele­vant game the­o­retic situ­a­tions. You don’t need to know the rules of Di­plo­macy to un­der­stand the post, but if you want a look you can find them here.

Study One: The Pri­soner’s Dilemma

The Pri­soner’s Dilemma is a clas­sic case in game the­ory in which two play­ers must de­cide whether or not to co­op­er­ate for a com­mon goal. If both play­ers co­op­er­ate, they both do bet­ter than if both defect, but one player can win big by defect­ing when the other co­op­er­ates. This situ­a­tion is at the heart of Di­plo­macy.

Ger­many and France have agreed to ally against Bri­tain. Both coun­tries have de­mil­i­ta­rized their mu­tual bor­der, and are con­cen­trat­ing all of their forces to the north, where they take province af­ter province of Bri­tish ter­ri­tory.

But Bri­tain is fight­ing back; not suc­cess­fully, but ev­ery inch of ter­ri­tory is hard-won. France is do­ing well for it­self and has cap­tured a few Bri­tish cities, but it could be do­ing bet­ter. The French player thinks to eir­self: I could ei­ther con­tinue bat­ter­ing against the heav­ily defended Bri­tish lines, or I could se­cretly ally with Bri­tain, stab Ger­many in the back, and waltz in along our un­defended mu­tual bor­der be­fore the Ger­mans even know what hit them. In­stead of fight­ing for each inch of Bri­tish land, I could be hav­ing din­ner in Ber­lin within a week.

Mean­while, in Ber­lin, the Ger­man player is look­ing to­wards France’s tempt­ingly un­defended bor­der and think­ing the ex­act same thing.

If both France and Ger­many are hon­or­able, and if both coun­tries know the other is hon­or­able, the two of them can con­tinue fight­ing Bri­tain with a two-to-one nu­mer­i­cal ad­van­tage and prob­a­bly di­vide England’s lu­cra­tive ter­ri­tory among the two of them.

If Ger­many is naively trust­ing and France is a dishon­est back­stab­ber, then France can get ob­scene re­wards by rol­ling over Ger­many while the Kaiser’s armies are tied up on the fields of England.

If both coun­tries are sus­pi­cious of the other, or if both coun­tries try to back­stab each other si­mul­ta­neously, then they will both di­vert forces away from the war on England to guard their mu­tual bor­der. They will not gain any ter­ri­tory in England, and they will not gain any ter­ri­tory along their bor­der. They’ve not only stabbed each other in the back, they’ve shot them­selves in the foot.

Study Two: Parfit’s Hitch-Hiker

The wiki de­scribes Derek Parfit’s fa­mous hitch­hiker prob­lem as:

Sup­pose you’re out in the desert, run­ning out of wa­ter, and soon to die—when some­one in a mo­tor ve­hi­cle drives up next to you. Fur­ther­more, the driver of the mo­tor ve­hi­cle is a perfectly self­ish ideal game-the­o­retic agent, and even fur­ther, so are you; and what’s more, the driver is Paul Ek­man, who’s re­ally, re­ally good at read­ing fa­cial microex­pres­sions. The driver says, “Well, I’ll con­vey you to town if it’s in my in­ter­est to do so—so will you give me $100 from an ATM when we reach town?”

Now of course you wish you could an­swer “Yes”, but as an ideal game the­o­rist your­self, you re­al­ize that, once you ac­tu­ally reach­town, you’ll have no fur­ther mo­tive to pay off the driver. “Yes,” you say. “You’re ly­ing,” says the driver, and drives off leav­ing you to die.

The so-called Key Lepanto open­ing is one of the more in­ter­est­ing open­ing strate­gies in Di­plo­macy, and one that re­quires guts of steel to pull off. It goes like this: Italy and Aus­tria de­cide to ally against Turkey. This is com­mon enough, and hin­dered by the fact that Turkey is prob­a­bly ex­pect­ing it and Italy’s kind of far away from Turkey any­way.

So Italy and Aus­tria do some­thing un­ex­pected. Italy swears loudly and pub­li­cly that ey’s al­lied with Aus­tria. Then, the first turn, Italy moves deep into un­defended Aus­trian ter­ri­tory! Aus­tria is in­censed, and curses loud and long at Italy’s be­trayal and at eir own stu­pidity for leav­ing the fron­tier un­guarded. Turkey laughs and leaves the two of them to their war when—boom—Aus­tria and Italy launch a co­or­di­nated at­tack against Turkey from Italy’s base deep in Aus­trian ter­ri­tory. The con­fused Turkey has no chance to or­ga­nize a re­sis­tance be­fore com­bined Italo-Aus­trian forces take Con­stantino­ple.

It’s fre­quently a suc­cess­ful strat­egy, es­pe­cially for Italy. You know what else is a suc­cess­ful strat­egy for Italy? Do­ing this up to the point where they take over lots of Aus­trian ter­ri­tory, for­get­ting the part where it was all just a ploy, and then end­ing up in con­trol of lots of Aus­trian ter­ri­tory, af­ter which they can fight Turkey at their leisure.

It’s very much in Italy’s ad­van­tage to play a Key Lepanto open­ing, and they may beg the Aus­trian player to go for it, say­ing cor­rectly that it would benefit both of them. But the Aus­trian player very of­ten re­fuses, tel­ling Italy that ey would have no in­cen­tive not to just keep the con­quered ter­ri­tory.

This prob­lem re­sem­bles the Hitch­hiker: Italy is the lost man, and Aus­tria is the driver. Italy re­ally wants Aus­tria to help em play the awe­some Key Lepanto open­ing, but Aus­tria knows that ey would have no in­cen­tive not to break his promise once Aus­tria’s given him the help he needs. As a re­sult, nei­ther coun­try gets what they want. The Key Lepanto open­ing is played only rarely, and this is one of the rea­sons.

Study Three: En­force­able Side Contracts

The Pri­soner’s Dilemma is non­triv­ial be­cause there’s no en­force­ment mechanism. In the pres­ence of an en­force­ment mechanism, it be­comes much sim­pler. Say two mob­sters are about to be ar­rested, and ex­pect to be put in a Pri­soner’s Dilemma type situ­a­tion. They ap­proach the mob boss with a con­tract with both of their names on it, say­ing that they have both agreed that if ei­ther of them tes­tifies against the other, the mob boss should send his goons to shoot the rat.

For many pay­off ma­tri­ces, sign­ing this con­tract will be a no-brainer. It en­sures your op­po­nent will co­op­er­ate at the rel­a­tively low cost of forc­ing you to co­op­er­ate your­self, and al­most guaran­tees you safe pas­sage into the de­sir­able (C,C) square. Not only does it pre­vent your op­po­nent doesn’t defect out of sheer greed, but it pre­vents your op­po­nent from wor­ry­ing that you’re go­ing to defect and then defect­ing em­self to save em­self from be­ing the chump.

The game of Di­plo­macy I won, I won through an en­force­able side con­tract (which lost me a friend and got me some ac­cu­sa­tions of cheat­ing, but this is par for the course for a good Di­plo­macy game). I was Bri­tain; my friend H was France. H and I knew each other from an me­dieval times role-play­ing game, in which we both held land and money. The me­dieval king­dom of this game had a law on the books that any oath wit­nessed by a no­ble was bind­ing on both par­ties and would be en­forced by the king. So H and I went into our role-play­ing game and swore an oath be­fore a co­op­er­a­tive no­ble, declar­ing that we would both aid each other in a per­ma­nent al­li­ance in Di­plo­macy, or else all our in-game lands and ti­tles would be forfeit.

A lot of peo­ple made fun of me for this, in­clud­ing H, but in my defense I did end up win­ning the game. H and I were able to do things that would oth­er­wise have been im­pos­si­ble; for ex­am­ple, in or­der to con­vince our en­emy Ger­many that we were at war, I took over the French city of Brest. Nor­mally, this would be al­most im­pos­si­ble for two al­lies to co­or­di­nate, even as a red her­ring, for ex­actly the rea­sons listed in the Hitch­hiker prob­lem above. Since the two of us were able to trust each other ab­solutely, this oth­er­wise difficult ma­neu­ver be­came easy.

One of the ad­van­tages to strong cen­tral gov­ern­ment is that it pro­vides an en­force­ment mechanism for con­tracts, which benefits all par­ties.

Study Four: Reli­gion As Enforcement

Reli­gion is a spe­cial case of the en­force­able side-con­tract in which God is do­ing the en­forc­ing. God doesn’t have to ex­ist for this to work; as long as at least one party be­lieves He does, the threat of pun­ish­ment will be cred­ible. The ad­van­tage of be­ing able to eas­ily make en­force­able side con­tracts even in the ab­sence of so­cial au­thor­ity may be one rea­son re­li­gion be­came so pop­u­lar, and if hu­mans do turn out to have a ge­netic ten­dency to­ward be­lief, the side con­tracts might have pro­vided part of the sur­vival ad­van­tage that spread the gene.

In a Youngstown Var­i­ant game (like Di­plo­macy, but with Eura­sia in­stead of just Europe), I was play­ing Italy and af­ter coloniz­ing Africa was try­ing to jug­gle my forces around to defend bor­ders with Ger­many, France, Turkey, and In­dia.

In­dia was played by my friend A, who I some­times have philo­soph­i­cal dis­cus­sions with and who I knew to be an arch-con­ser­va­tive re­li­gion-and-fam­ily-val­ues type. I de­cided to try some­thing which, as far as I know, no one’s ever tried in a Di­plo­macy game be­fore. “Do you swear in the name of God and your sa­cred honor that you won’t at­tack me?” I asked.

“Yes,” said A, and I knew he meant it, be­cause he takes that sort of thing re­ally se­ri­ously. I don’t know if he thought he would liter­ally go to Hell if he broke his oath, but I’m pretty sure he wasn’t will­ing to risk it over a board game. So I de­mil­i­ta­rized my bor­der with In­dia. I con­cen­trated my forces to the west, he con­cen­trated them to the east, and both avoided a costly stale­mate in the In­dian Ocean and had more forces to send el­se­where. In the fu­ture, I will seek out A for al­li­ances more of­ten, since I have ex­tra rea­son to be­lieve he won’t be­tray me; this will put A in an un­usu­ally strong po­si­tion.

This is not a unique ad­van­tage of re­li­gion; any strongly held philos­o­phy that trumps self-in­ter­est would do. I would have made the same deal with Ali­corn, who has stated loudly and pub­li­cly that she is a de­on­tol­o­gist who has a deep per­sonal aver­sion to ly­ing2. I would have made it with Eliezer, who has a con­se­quen­tial­ist moral­ity but, on ac­count of the con­se­quences, has said he would not break an oath even for the sake of sav­ing the world.

But I only trust Ali­corn and Eliezer be­cause I’ve dis­cussed moral­ity with both of them in a situ­a­tion where they had no in­cen­tive to lie; it was only in the very un­usual con­di­tions of Less Wrong that they could send such a sig­nal be­liev­ably. Reli­gion is a much eas­ier sig­nal to send and re­ceive with­out be­ing a moral philoso­pher.

Study Five: Ex­cuses as De­vi­a­tions from a Rule

My pre­vi­ous post, Eight Short Stud­ies on Ex­cuses, was in­spired by a ma­neu­ver I pul­led dur­ing a Di­plo­macy game.

I was Italy, and Turkey and I had formed a mu­tual al­li­ance against Aus­tria. As part of the al­li­ance, we had de­cided not to fight over who got the lu­cra­tive neu­tral ter­ri­to­ries in be­tween our em­pires. I would get Egypt, Turkey would get Greece and Ye­men, and we would avoid the re­source drain of fight­ing each other for them so we could both con­cen­trate on Aus­tria.

Both Turkey and I would have liked to grab the cen­ters that had been promised to the other. But both Turkey and I knew that main­tain­ing the gen­eral rule of al­li­ance be­tween us was higher util­ity than get­ting one ex­tra ter­ri­tory. BUT both Turkey and I knew that the other would be loathe to break off the al­li­ance be­tween just be­cause their part­ner had com­mit­ted one lit­tle in­frac­tion. BUT both Turkey and I knew that we would have to do ex­actly that, or else our ally would have a carte blanche to vi­o­late what­ever terms of the al­li­ance they wanted.

Then In­dia (from whom I had not yet ex­tracted his oath) made a move to­wards Ye­men, threat­en­ing to take it from both of us. I re­sponded by mov­ing a navy to Ye­men, sup­pos­edly to see off the In­dian men­ace. I then mes­saged Turkey, say­ing that al­though I still re­spected the terms of our al­li­ance, he was clearly too weak to keep Ye­men out of In­dian hands, so I would be for­tify­ing it for him, and I hoped he would have the ma­tu­rity to see this as a mu­tu­ally benefi­cial move to pre­vent In­dian ex­pan­sion­ism, and not get too hung up on the ex­act terms of our al­li­ance.

The gam­bit worked: Turkey de­cided that main­tain­ing our al­li­ance was more im­por­tant than keep­ing Ye­men, and that be­cause of the trou­ble with In­dia my con­quest of Ye­men was not in­dica­tive of a gen­eral pat­tern of al­li­ance-break­ing that needed to be pun­ished.

I can’t claim to­tal vic­tory here: sev­eral years later, when the threat of Aus­tria had dis­ap­peared, Turkey be­trayed me and cap­tured half my em­pire, partly be­cause of my ac­tions in Ye­men.

Study Six: For the Sake of Revenge

This comes from the book Game The­ory at Work:

Con­sider the emo­tion of re­venge. At its core, re­venge means hurt­ing some­one who has harmed you, even if you would be bet­ter off leav­ing him alone. Re­venge is an ir­ra­tional de­sire to harm oth­ers who have in­jured our loved ones or us.

To see the benefit of be­ing known as venge­ful, con­sider a small com­mu­nity liv­ing in pre­his­toric times. Imag­ine that a group of raiders stole food from this com­mu­nity. A ra­tio­nal com­mu­nity would hunt down the raiders only if the cost of do­ing so was not too high. A ven­gence-en­dowed com­mu­nity would hunt down the raiders re­gard­less of the cost. Since the raiders would rather go af­ter the ra­tio­nal com­mu­nity, be­ing per­ceived as venge­ful pro­vides you with pro­tec­tion and there­fore con­fers an evolu­tion­ary ad­van­tage.

I play Di­plo­macy of­ten against the same peo­ple, so I de­cided I needed to cul­ti­vate a rep­u­ta­tion for venge­ful­ness. And by “de­cided to cul­ti­vate a rep­u­ta­tion for venge­ful­ness”, I mean “Turkey be­trayed me and I was filled with the burn­ing rage of a thou­sand suns”.

So my drive for re­venge was mostly emo­tional in­stead of ra­tio­nal. But what I didn’t do was sup­press my anger, the way peo­ple are always tel­ling you. Sup­press­ing anger is a use­ful strat­egy for one-shot games, but in an iter­ated game, get­ting a rep­u­ta­tion for anger is of­ten more valuable than be­hav­ing in your im­me­di­ate ra­tio­nal self-in­ter­est.

So I de­cided to throw the game to Ger­many, Turkey’s biggest ri­val. I moved my forces away from the Ital­ian-Ger­man bor­der and in­vited Ger­many to take over my ter­ri­tory. At the same time, I used my re­main­ing forces sup­port­ing Ger­man at­tacks against Turkey. The Aus­tri­ans, who had been deal­ing with Turkey’s be­tray­als even longer than I had, hap­pily joined in. With our help, Ger­man forces scored sev­eral re­sound­ing vic­to­ries against Turkey and pushed it back from near the top of the game down to a dis­tant third.

Around the same time, Ger­many’s other en­emy France also be­trayed me. So I told France I was throw­ing the game to Ger­many to pun­ish him. No point in miss­ing a perfectly good op­por­tu­nity to cul­ti­vate a rep­u­ta­tion for venge­ful­ness.

If I had done the ra­tio­nal thing and ex­cused Turkey’s be­trayal be­cause it was in my self-in­ter­est to cut my losses, I could have had a mediocre end game, and Turkey’s player would have hap­pily be­trayed me the next game as soon as he saw any ad­van­tage in do­ing so. In­stead, I’m do­ing very poorly in the end game, but Turkey—and ev­ery­one else—will be very wary about be­tray­ing me next time around.

Study Seven: In-Group Bias as a Schel­ling Point

I made the mis­take of mod­er­at­ing a game of Di­plo­macy at the SIAI House, which turned into one of the worst I’ve ever seen. The play­ers were five SIAI Visit­ing Fel­lows and two of my non-SIAI friends who hap­pened to be in the area.

Jasen came up with the idea of an al­li­ance of the five SIAI play­ers against my two friends. Although a few of the Fel­lows vac­illated back and forth and defected a few times, he was gen­er­ally able to keep the loy­alty of the five Fel­lows un­til my two friends had been elimi­nated from the game rel­a­tively early on. Although nor­mally the game would have con­tinued un­til one of the Fel­lows man­aged to dom­i­nate the oth­ers, it was already very late and we called it a night at that point.

It’s easy to ex­plain what hap­pened as an ir­ra­tional in-group bias, or as “loy­alty” or “pa­tri­o­tism” among the SIAI folk. Jasen him­self ex­plained it as a de­sire to prove that SIAI peo­ple were es­pe­cially co­op­er­a­tive and es­pe­cially good at game the­ory, which I sup­pose worked. But there’s an­other, com­pletely the­o­ret­i­cal per­spec­tive from which to view the SIAI Alli­ance.

Imag­ine you are on a life­boat with nine other peo­ple, and de­ter­mine that one of the ten of you must be kil­led and eaten to provide sus­te­nance to the oth­ers. You are all ready to draw lots to de­cide who is din­ner when you shout out “Hey, in­stead of this whole draw­ing lots thing, let’s kill and eat Bob!”

If your fel­low cast­aways are ra­tio­nal agents, they might just agree. If they go with lots, each has a 10% chance of end­ing up din­ner. If ev­ery­one just agrees on Bob, then ev­ery­one has a 0% chance of end­ing up din­ner (ex­cept poor Bob). Nine out of ten peo­ple are bet­ter off, and nine out of ten of you vote to adopt the new plan. Whether your life­boat de­cides things by ma­jor­ity vote or by phys­i­cal vi­o­lence, it doesn’t look good for Bob.

But imag­ine a week later, you still haven’t been res­cued, and the whole situ­a­tion re­peats. If ev­ery­one lets you re­peat your ac­tion of call­ing out a name, there’s a 19 chance it’ll be eir name—no bet­ter than draw­ing lots. In fact, since you’re very un­likely to call out your own name, it’s more of a 18 chance—worse than just draw­ing lots. So ev­ery­one would like to be the one who calls out the name, and as soon as the lots are taken out, ev­ery­one shouts “Hey, in­stead of the whole draw­ing lots thing, let’s kill and eat X!” where X is a differ­ent per­son for each of the nine cast­aways. This is ut­terly use­less, and you prob­a­bly end up just draw­ing lots.

But sup­pose eight of the nine of you are blond, and one is a brunette. The brunette is now a Schel­ling point. If you choose to kill and eat the brunette, there’s a pretty good chance all of your blond friends will do the same, even if none of you had a pre-ex­ist­ing prej­u­dice against brunettes. There­fore, all eight of you shout out “Let’s kill and eat the brunette!”, since this is safer than draw­ing lots. Your life­boat has in­vented in-group bias from ra­tio­nal prin­ci­ples.

Such al­li­ances are equally at­trac­tive in Di­plo­macy. When the five SIAI Fel­lows al­lied against my two friends, they en­sured there was a five-against-two al­li­ance with them­selves on the win­ning side, and suc­cess­fully re­duced the game­board from six op­po­nents to four. Although they could have done this with any­one (eg Jasen could have se­lected two other Fel­lows and my two friends, and forged an equiv­a­lent coal­i­tion of five), Jasen would have been at risk of five other peo­ple hav­ing the same idea and ex­clud­ing him. By choos­ing a nat­u­ral and ob­vi­ous di­vi­sion in which he was on the ma­jor­ity, Jasen avoided this risk.

Ra­tion­al­ist Diplomacy

I’m in­ter­ested in see­ing what a Di­plo­macy game be­tween Less Wrongers looks like. I’m will­ing to mod­er­ate. The first seven peo­ple to sign up get places (don’t sign up if you don’t ex­pect to have enough time for about two or three turns/​week), and the next few can be al­ter­nates. Doesn’t mat­ter if you’ve ever played be­fore as long as you read the rules above and think you un­der­stand them. (We already have seven peo­ple. See the post in Dis­cus­sion. If many more sign up, some­one else may want to mod­er­ate a sec­ond game).


1: Source: “Nice Guys Finish First” in the Frameshift album Un­weav­ing the Rain­bow.

2. Ali­corn wishes me to note that she con­sid­ers any­one play­ing a Di­plo­macy game with­out prior out-of-game-con­text agree­ments se­cured to have waived eir right to com­plete hon­esty from her, but the gen­eral prin­ci­ple still stands.