I really like this post. I think it points out an important problem with intuitive credit-assignment algorithms which people often use. The incentive toward inaction is a real problem which is often encountered in practice. While I was somewhat aware of the problem before, this post explains it well.
I also think this post is wrong, in a significant way: asymmetric justice is not always a problem and is sometimes exactly what you want. in particular, it’s how you want a justice system (in the sense of police, judges, etc) to work.
The book Law’s Order explains it like this: you don’t want theft to be punished in keeping with its cost. Rather, in order for the free market to function, you want theft to be punished harshly enough that theft basically doesn’t happen.
Zvi speaks as if the purpose of the justice system is to reward positive externalities and punish negative externalities, to align everyone’s incentives. While this is a noble goal, Law’s Order sees it as a goal to be taken care of by other parts of society, in particular the free market. (Law’s Order is a fairly libertarian book, so it puts a lot of faith in the free market.)
The purpose of the justice system is to enforce the structure such that those other institutions can do their jobs. The free market can’t optimize people’s lives properly if theft and murder are a constant and contracts cannot be enforced.
So, it makes perfect sense for a justice system to be asymmetric. Its role is to strongly disincentivize specific things, not to broadly provide compensatory incentives.
(For this reason, scales are a pretty terrible symbol for justice.)
In general, we might conclude that credit assignment systems need two parts:
A “symmetric” part, which attempts to allocate credit in as calibrated a way as it can, rewarding good work and punishing bad.
An “asymmetric” part, which harshly enforces the rules which ensure that the symmetric part can function, ensuring that those rules are followed frequently enough for things to function.
This also gives us a criterion for when punishment should be disproportionate: only those things which interfere with the more proportionate credit assignment should be disproportionately punished.
Overall, I still think this is a great post, I just think there’s more to the issue.
I really like this post. I think it points out an important problem with intuitive credit-assignment algorithms which people often use. The incentive toward inaction is a real problem which is often encountered in practice. While I was somewhat aware of the problem before, this post explains it well.
Rereading this, one thought that comes to mind is that Copenhagen ethics and asymmetric justice may be another side of blackbox reinforcement learning driven by egalitarianism. Just as a CEO is held strictly responsible for everything that happens under them and is punished, regardless of whether we reasonably believe the bad results were not their fault, because we are insufficiently sure of judging fault and cannot observe all the actions the CEO did or did not do; or anyone who keeps a tiger in their backyard is held 100% responsible when that tiger eats someone no matter how much they swear they thought the fences were adequate; anyone who gets involved with a problem and doesn’t meet some high bar is automatically assumed to be guilty, because we can’t be sure they didn’t do some skulduggery or gossip, so if they benefit in any way from the problem, we especially want to punish them just to be safe.
In a large complex world with billions of people where win-win exchanges are universal and where there are power law payoffs and it is (far) more important to work smarter than harder and sheer pain has only the most tenuous relationship to how valuable that labor is to billions of other people, this is a ridiculous heuristic which has pernicious consequences. But in a tribe or village, struggling for egalitarianism and to counter the dominance of would-be big men, where the goal is to maintain the status quo and ‘progress’ is a meaningless word and everyone knows that in every transaction there is a winner and a loser, then asymmetric justice just feels right. (Does Amazon benefit in any way from hiring the homeless? Then maybe there’s a sinister Amazonian conspiracy to create homeless just to buy Bezos a new yacht—who can be sure that Amazon didn’t somehow cause or contribute to it? Something something gentrification small businesses amirite? Perfidious Amazon! Anyway, if Amazon wants to hire them, that’s just further proof that it’s exploiting the homeless, because why would Amazon want to be the loser in the transaction? Therefore, the homeless must be the losers, and only the most egregiously evil would seek to exploit the homeless like that! The homeless would only benefit if the noble People passed a law to force Amazon to hire them at ‘fair’ wages, ensuring they are the winners in the transaction.)
I think you go too far by also postulating that (in the evolutionary past) it would be natural to assume that every game is zero-sum. There are clearly a lot of cooperative interactions in that kind of environment. Every interaction has a ‘winner’ and a ‘loser’ because of the focus on egalitarianism: the ‘loser’ is the one who got the worse end of the deal (according to the partly-understood, partly-hypothetical ideal of fairness). Ganging up on whoever keeps getting the best side of deals is a natural way to enforce fair splits.
Which seems different from the involvement heuristic you mention. The involvement heuristic (EG, blame the CEO for anything the company does) has no obvious reason to be asymmetric. It seems dumb. If we’re not sure how to assign credit, punishing everyone involved seems to go hand in hand with rewarding everyone involved.
So I would still think the main reason for asymmetric justice is coordination around norms (such as fairness norms) that should almost always be followed. It doesn’t make sense to reward people for fairness if almost everyone is supposed to be fair almost all of the time. It makes far more sense to punish the unfair.
So, yeah, then when you couple that with the involvement heuristic… you get copenhagen-ethics.
I really like this post. I think it points out an important problem with intuitive credit-assignment algorithms which people often use. The incentive toward inaction is a real problem which is often encountered in practice. While I was somewhat aware of the problem before, this post explains it well.
I also think this post is wrong, in a significant way: asymmetric justice is not always a problem and is sometimes exactly what you want. in particular, it’s how you want a justice system (in the sense of police, judges, etc) to work.
The book Law’s Order explains it like this: you don’t want theft to be punished in keeping with its cost. Rather, in order for the free market to function, you want theft to be punished harshly enough that theft basically doesn’t happen.
Zvi speaks as if the purpose of the justice system is to reward positive externalities and punish negative externalities, to align everyone’s incentives. While this is a noble goal, Law’s Order sees it as a goal to be taken care of by other parts of society, in particular the free market. (Law’s Order is a fairly libertarian book, so it puts a lot of faith in the free market.)
The purpose of the justice system is to enforce the structure such that those other institutions can do their jobs. The free market can’t optimize people’s lives properly if theft and murder are a constant and contracts cannot be enforced.
So, it makes perfect sense for a justice system to be asymmetric. Its role is to strongly disincentivize specific things, not to broadly provide compensatory incentives.
(For this reason, scales are a pretty terrible symbol for justice.)
In general, we might conclude that credit assignment systems need two parts:
A “symmetric” part, which attempts to allocate credit in as calibrated a way as it can, rewarding good work and punishing bad.
An “asymmetric” part, which harshly enforces the rules which ensure that the symmetric part can function, ensuring that those rules are followed frequently enough for things to function.
This also gives us a criterion for when punishment should be disproportionate: only those things which interfere with the more proportionate credit assignment should be disproportionately punished.
Overall, I still think this is a great post, I just think there’s more to the issue.
Rereading this, one thought that comes to mind is that Copenhagen ethics and asymmetric justice may be another side of blackbox reinforcement learning driven by egalitarianism. Just as a CEO is held strictly responsible for everything that happens under them and is punished, regardless of whether we reasonably believe the bad results were not their fault, because we are insufficiently sure of judging fault and cannot observe all the actions the CEO did or did not do; or anyone who keeps a tiger in their backyard is held 100% responsible when that tiger eats someone no matter how much they swear they thought the fences were adequate; anyone who gets involved with a problem and doesn’t meet some high bar is automatically assumed to be guilty, because we can’t be sure they didn’t do some skulduggery or gossip, so if they benefit in any way from the problem, we especially want to punish them just to be safe.
In a large complex world with billions of people where win-win exchanges are universal and where there are power law payoffs and it is (far) more important to work smarter than harder and sheer pain has only the most tenuous relationship to how valuable that labor is to billions of other people, this is a ridiculous heuristic which has pernicious consequences. But in a tribe or village, struggling for egalitarianism and to counter the dominance of would-be big men, where the goal is to maintain the status quo and ‘progress’ is a meaningless word and everyone knows that in every transaction there is a winner and a loser, then asymmetric justice just feels right. (Does Amazon benefit in any way from hiring the homeless? Then maybe there’s a sinister Amazonian conspiracy to create homeless just to buy Bezos a new yacht—who can be sure that Amazon didn’t somehow cause or contribute to it? Something something gentrification small businesses amirite? Perfidious Amazon! Anyway, if Amazon wants to hire them, that’s just further proof that it’s exploiting the homeless, because why would Amazon want to be the loser in the transaction? Therefore, the homeless must be the losers, and only the most egregiously evil would seek to exploit the homeless like that! The homeless would only benefit if the noble People passed a law to force Amazon to hire them at ‘fair’ wages, ensuring they are the winners in the transaction.)
I think you go too far by also postulating that (in the evolutionary past) it would be natural to assume that every game is zero-sum. There are clearly a lot of cooperative interactions in that kind of environment. Every interaction has a ‘winner’ and a ‘loser’ because of the focus on egalitarianism: the ‘loser’ is the one who got the worse end of the deal (according to the partly-understood, partly-hypothetical ideal of fairness). Ganging up on whoever keeps getting the best side of deals is a natural way to enforce fair splits.
Which seems different from the involvement heuristic you mention. The involvement heuristic (EG, blame the CEO for anything the company does) has no obvious reason to be asymmetric. It seems dumb. If we’re not sure how to assign credit, punishing everyone involved seems to go hand in hand with rewarding everyone involved.
So I would still think the main reason for asymmetric justice is coordination around norms (such as fairness norms) that should almost always be followed. It doesn’t make sense to reward people for fairness if almost everyone is supposed to be fair almost all of the time. It makes far more sense to punish the unfair.
So, yeah, then when you couple that with the involvement heuristic… you get copenhagen-ethics.
Sucks.