I really like this post. I think it points out an important problem with intuitive credit-assignment algorithms which people often use. The incentive toward inaction is a real problem which is often encountered in practice. While I was somewhat aware of the problem before, this post explains it well.
Rereading this, one thought that comes to mind is that Copenhagen ethics and asymmetric justice may be another side of blackbox reinforcement learning driven by egalitarianism. Just as a CEO is held strictly responsible for everything that happens under them and is punished, regardless of whether we reasonably believe the bad results were not their fault, because we are insufficiently sure of judging fault and cannot observe all the actions the CEO did or did not do; or anyone who keeps a tiger in their backyard is held 100% responsible when that tiger eats someone no matter how much they swear they thought the fences were adequate; anyone who gets involved with a problem and doesn’t meet some high bar is automatically assumed to be guilty, because we can’t be sure they didn’t do some skulduggery or gossip, so if they benefit in any way from the problem, we especially want to punish them just to be safe.
In a large complex world with billions of people where win-win exchanges are universal and where there are power law payoffs and it is (far) more important to work smarter than harder and sheer pain has only the most tenuous relationship to how valuable that labor is to billions of other people, this is a ridiculous heuristic which has pernicious consequences. But in a tribe or village, struggling for egalitarianism and to counter the dominance of would-be big men, where the goal is to maintain the status quo and ‘progress’ is a meaningless word and everyone knows that in every transaction there is a winner and a loser, then asymmetric justice just feels right. (Does Amazon benefit in any way from hiring the homeless? Then maybe there’s a sinister Amazonian conspiracy to create homeless just to buy Bezos a new yacht—who can be sure that Amazon didn’t somehow cause or contribute to it? Something something gentrification small businesses amirite? Perfidious Amazon! Anyway, if Amazon wants to hire them, that’s just further proof that it’s exploiting the homeless, because why would Amazon want to be the loser in the transaction? Therefore, the homeless must be the losers, and only the most egregiously evil would seek to exploit the homeless like that! The homeless would only benefit if the noble People passed a law to force Amazon to hire them at ‘fair’ wages, ensuring they are the winners in the transaction.)
I think you go too far by also postulating that (in the evolutionary past) it would be natural to assume that every game is zero-sum. There are clearly a lot of cooperative interactions in that kind of environment. Every interaction has a ‘winner’ and a ‘loser’ because of the focus on egalitarianism: the ‘loser’ is the one who got the worse end of the deal (according to the partly-understood, partly-hypothetical ideal of fairness). Ganging up on whoever keeps getting the best side of deals is a natural way to enforce fair splits.
Which seems different from the involvement heuristic you mention. The involvement heuristic (EG, blame the CEO for anything the company does) has no obvious reason to be asymmetric. It seems dumb. If we’re not sure how to assign credit, punishing everyone involved seems to go hand in hand with rewarding everyone involved.
So I would still think the main reason for asymmetric justice is coordination around norms (such as fairness norms) that should almost always be followed. It doesn’t make sense to reward people for fairness if almost everyone is supposed to be fair almost all of the time. It makes far more sense to punish the unfair.
So, yeah, then when you couple that with the involvement heuristic… you get copenhagen-ethics.
Rereading this, one thought that comes to mind is that Copenhagen ethics and asymmetric justice may be another side of blackbox reinforcement learning driven by egalitarianism. Just as a CEO is held strictly responsible for everything that happens under them and is punished, regardless of whether we reasonably believe the bad results were not their fault, because we are insufficiently sure of judging fault and cannot observe all the actions the CEO did or did not do; or anyone who keeps a tiger in their backyard is held 100% responsible when that tiger eats someone no matter how much they swear they thought the fences were adequate; anyone who gets involved with a problem and doesn’t meet some high bar is automatically assumed to be guilty, because we can’t be sure they didn’t do some skulduggery or gossip, so if they benefit in any way from the problem, we especially want to punish them just to be safe.
In a large complex world with billions of people where win-win exchanges are universal and where there are power law payoffs and it is (far) more important to work smarter than harder and sheer pain has only the most tenuous relationship to how valuable that labor is to billions of other people, this is a ridiculous heuristic which has pernicious consequences. But in a tribe or village, struggling for egalitarianism and to counter the dominance of would-be big men, where the goal is to maintain the status quo and ‘progress’ is a meaningless word and everyone knows that in every transaction there is a winner and a loser, then asymmetric justice just feels right. (Does Amazon benefit in any way from hiring the homeless? Then maybe there’s a sinister Amazonian conspiracy to create homeless just to buy Bezos a new yacht—who can be sure that Amazon didn’t somehow cause or contribute to it? Something something gentrification small businesses amirite? Perfidious Amazon! Anyway, if Amazon wants to hire them, that’s just further proof that it’s exploiting the homeless, because why would Amazon want to be the loser in the transaction? Therefore, the homeless must be the losers, and only the most egregiously evil would seek to exploit the homeless like that! The homeless would only benefit if the noble People passed a law to force Amazon to hire them at ‘fair’ wages, ensuring they are the winners in the transaction.)
I think you go too far by also postulating that (in the evolutionary past) it would be natural to assume that every game is zero-sum. There are clearly a lot of cooperative interactions in that kind of environment. Every interaction has a ‘winner’ and a ‘loser’ because of the focus on egalitarianism: the ‘loser’ is the one who got the worse end of the deal (according to the partly-understood, partly-hypothetical ideal of fairness). Ganging up on whoever keeps getting the best side of deals is a natural way to enforce fair splits.
Which seems different from the involvement heuristic you mention. The involvement heuristic (EG, blame the CEO for anything the company does) has no obvious reason to be asymmetric. It seems dumb. If we’re not sure how to assign credit, punishing everyone involved seems to go hand in hand with rewarding everyone involved.
So I would still think the main reason for asymmetric justice is coordination around norms (such as fairness norms) that should almost always be followed. It doesn’t make sense to reward people for fairness if almost everyone is supposed to be fair almost all of the time. It makes far more sense to punish the unfair.
So, yeah, then when you couple that with the involvement heuristic… you get copenhagen-ethics.
Sucks.