The Prisoner’s Dilemma is a well-studied game in game theory, where supposedly rational incentive following leads to both players stabbing each other in the back and being worse off than if they had cooperated.
The original formulation, via Wikipedia:
Two members of a criminal gang are arrested and imprisoned. Each prisoner is in solitary confinement with no means of communicating with the other. The prosecutors lack sufficient evidence to convict the pair on the principal charge, but they have enough to convict both on a lesser charge. Simultaneously, the prosecutors offer each prisoner a bargain. Each prisoner is given the opportunity either to betray the other by testifying that the other committed the crime, or to cooperate with the other by remaining silent. The possible outcomes are:
If A and B each betray the other, each of them serves two years in prison
If A betrays B but B remains silent, A will be set free and B will serve three years in prison
If A remains silent but B betrays A, A will serve three years in prison and B will be set free
If A and B both remain silent, both of them will serve only one year in prison (on the lesser charge).
The “stay silent” option is generally called Cooperate, and the “betray” option is called Defect. The only Nash Equilibrium of the Prisoner’s Dilemma is both players defecting, even though each would prefer the cooperate/cooperate outcome.
Notice that it’s only if you treat the other player’s decision as completely independent from yours, if the other player defects, then you score higher if you defect as well, whereas if the other player cooperates, you do better by defecting. Hence Nash Equilibrium to defect (at least if the game is to be played only once), and indeed, this is what classical causal decision theory says. And yet—and yet, if only somehow both players could agree to cooperate, they would both do better than if they both defected. If the players are timeless decision agents, or functional decision theory agents, they can.
A popular variant is the Iterated Prisoner’s Dilemma, where two agents play the Prisoner’s Dilemma against each other a number of times in a row. A simple and successful strategy is called Tit for Tat—cooperate on the first round, then on subsequent rounds do whatever your opponent did on the last round.
Prisoner’s dilemma (Stanford Encyclopedia of Philosophy)
Drescher, Gary (2006). Good and Real. Cambridge: The MIT Press. ISBN 0262042339.