Yeah, I think that sort of presentation is anti-useful for understanding the world, since it’s picking a rather arbitrary mathematical theory and just insisting “this is what rational people do”, without getting people to think it through and understand why or if that’s actually true.
The reason a rational agent will likely defect in a realistic prisoner’s dilemma against a normal human is because it believes the human’s actions to be largely uncorrelated with its own, since it doesn’t have a good enough model of the human’s mind to know how it thinks. (And the reason why humans defect is the same, with the added obstacle that the human isn’t even rational themselves.)
Teaching that rational agents defect because that’s the Nash equilibrium and rational agents always go to the Nash equilibrium is just an incorrect model of rationality, and agents that are actually rational can consistently win against Nash-seekers.
Yeah, I think that sort of presentation is anti-useful for understanding the world, since it’s picking a rather arbitrary mathematical theory and just insisting “this is what rational people do”, without getting people to think it through and understand why or if that’s actually true.
The reason a rational agent will likely defect in a realistic prisoner’s dilemma against a normal human is because it believes the human’s actions to be largely uncorrelated with its own, since it doesn’t have a good enough model of the human’s mind to know how it thinks. (And the reason why humans defect is the same, with the added obstacle that the human isn’t even rational themselves.)
Teaching that rational agents defect because that’s the Nash equilibrium and rational agents always go to the Nash equilibrium is just an incorrect model of rationality, and agents that are actually rational can consistently win against Nash-seekers.