The Game Theory of Blackmail

This blog post is com­posed as fol­low­ing:

  1. Re­view of Pri­son­ers Dilemma

  2. Ex­pla­na­tion of Game of Chicken by com­par­ing it to Pri­son­ers Dilemma

  3. Black­mail is a Game of Chicken

  4. Why we should care about black­mail/​Game of Chicken

  5. What to do? Iter­ated Game of Chicken?

You are en­couraged to skip ahead to the part that in­ter­ests you

1. Re­view of Pri­son­ers Dilemma

Pri­son­ers Dilemma is a class of two player games which can rep­re­sent for ex­am­ple mu­tual benefi­cial co­op­er­a­tion, or the tragedy of the com­mons. I don’t think it is con­tro­ver­sial to say that this class of games are im­por­tant in al­most any multi-agent sce­nario.

In a Pri­son­ers Dilemma , each player gets to choose be­tween two ac­tions, usu­ally called “co­op­er­ate” and “defect”. Fur­ther more the pay­offs haved to fulfill the fol­low­ing:

  • Hold­ing my ac­tion con­stant it is bet­ter for me if you co­op­er­ate.

  • Hold­ing your ac­tion con­stant, it is bet­ter for me if I defect.

  • Co­op­er­ate-co­op­er­ate is Pareto op­ti­mal (even when in­clud­ing mixed strate­gies).

Ex­am­ple of a pay­out ma­trix for Pri­son­ers Dilemma:

In this par­tic­u­lar ex­am­ple, co­op­er­ate cor­re­sponds to spend­ing one of your own util­ity to give the other player two util­ity, and defect cor­re­sponds to do­ing noth­ing. This can rep­re­sent a situ­a­tion with the pos­si­bil­ity of mu­tual benefit from co­op­er­a­tion, but where it is pos­si­ble to win even more (at the other play­ers ex­pense) by cheat­ing.

But we can also con­sider a nega­tive game:

Here co­op­er­ate is do­ing noth­ing, while defect cor­re­sponds to gain­ing one util­ity for your­self while cost­ing the other player two util­ity. This can rep­re­sent burn­ing the com­mons (if the play­ers defect) or not (if they co­op­er­ate).

2. Ex­pla­na­tion of Game of Chicken by com­par­ing it to Pri­son­ers Dilemma

Just like Pri­son­ers Dilemma, Game of Chicken is a two player game, where each player can choose be­tween two ac­tions. Th­ese ac­tions are typ­i­cally called “swerve” and “straight”, but in this blog post I will in­stead call the two ac­tions “co­op­er­ate” and “defect” as to more eas­ily com­pare with Pri­son­ers Dilemma.

Also the same as Pri­son­ers Dilemma: In Game of Chicken, I get the best pay­out if I defect and you co­op­er­ate (and vice versa). The differ­ence is that con­di­tional on you defect­ing, it is bet­ter for me if i co­op­er­ate.

A two ac­tion, two player game is a Game of Chicken if:

  • Hold­ing my ac­tion con­stant it is bet­ter for me if you co­op­er­ate.

  • If you co­op­er­ate it is bet­ter for me if I defect.

  • If you defect is bet­ter for me to co­op­er­ate.

  • Co­op­er­ate-co­op­er­ate is Pareto op­ti­mal (even when in­clud­ing mixed strate­gies).

Fur­ther­more, defect-defect is tra­di­tion­ally su­per bad for both play­ers. But I would not say that this is a nec­es­sary con­di­tion for some­thing to be a Game of Chicken.

Ex­am­ple pay­off ma­trix:

The in­ter­est­ing part here is that I can pres­sure you to co­op­er­ate by cred­ibly con­vinc­ing you that I will defect. In other words, there is a first mover ad­van­tage, the first one to pre­com­mit to defect­ing will win against a ra­tio­nal player. How­ever, this fact is of course known by ev­ery ra­tio­nal agent, so it might be a ra­tio­nal move to pre-com­mit to always defect in such games, no mater what. Then again, if two play­ers with such com­mit­ments meet, they will both lose.

3. Black­mail is a Game of Chicken

I think that this is eas­iest ex­plained by just writ­ing out an ex­am­ple pay­out matrix

If the the black­mailed player gives in, then they pay two util­ity to give the other player one util­ity. If the black­mailed player doesn’t give in the black­mailer will carry out the threat which is cost­ing both play­ers ten util­ity. If the the black­mailer doesn’t ac­tu­ally black­mails, than noth­ing hap­pens.

Com­pare this to the ex­am­ple pay­out ma­trix of Game of Chicken. The black­mail pay­out ma­trix is not ex­actly the same, but I claim that in essence this is the same game. If you can han­dle Game of Chicken then you can han­dle black­mail both as the black­mailer and the black­mailed.

Not all black­mail is a Game of Chicken. If there is not cost in car­ry­ing out the threat then we are in a differ­ent type of situ­a­tion. How­ever I ex­pect this to be rare. It seems un­likely to me that there is no op­por­tu­nity cost at all in car­ry­ing out the threat. Fur­ther more, even if costless threats ex­ists in some situ­a­tions this does not in­val­i­date the ar­gu­ment for con­sid­er­ing those black­mail situ­a­tion where there is a cost to the black­mailer to car­ry­out the threat.

If the black­mailer gains util­ity by car­ry­ing out the threat then I would ar­gue that it is not ex­actly black­mail any­more. If I have an ac­tion that I can take that would help me but hurt you and I ask you for some com­pen­sa­tion for re­frain­ing from tak­ing this ac­tion, then this is more like a value trade than a black­mail.

4. Why we should care about black­mail/​Game of Chicken

Pri­son­ers Dilemma re­ceives a lot of at­ten­tion be­cause this class of games rep­re­sents an im­por­tant type of situ­a­tion in most mul­ti­player en­vi­ron­ments. I claim that this is also true for Game of Chicken.

In any situ­a­tion where one agent (A) has the abil­ity to use up some of its own re­sources to im­pose a cost on an­other agent (B), then A can choose to black­mail B, thus cre­at­ing a Game of Chicken like situ­a­tion. And if A thinks that it can win this game, then it will be tempted to en­gage in black­mail.

If you ex­pect that:

  • It is im­por­tant to build AI’s that can act well in multi-agent situ­a­tions (e.g. be­cause there will be sev­eral si­mul­ta­neous AIs that are similarly pow­er­ful, or there will be acausal trade and threats be­tween agents in differ­ent uni­verses simu­lat­ing each other)

and

  • Toy model such as Pri­son­ers Dilemma are useful

then you should also care about Game of Chicken.

5. What to do? Iter­ated Game of Chicken?

What should we do about these in­sights? I am not sure yet. But one pos­si­ble di­rec­tions is to study iter­ated Game of Chicken.

Abram Dem­ski ar­gues that In Log­i­cal Time, All Games are Iter­ated Games. Ba­si­cally if agents are simu­lat­ing each other then this is sort of equiv­a­lent to the agent play­ing an iter­ated game.

Ques­tion for the com­ment sec­tion: What would be the win­ning strat­egy in iter­ated Game of Chicken?

I might run a tour­na­ment with differ­ent strate­gies.


This post was writ­ten with the sup­port of the EA Hotel