There is a mistake in updating beliefs around norms that I have seen multiple times in different contexts made by different people. This is my attempt to classify this (fallacy? /mistake?), in order to give it a name and make it easier to refer to.
A friend of mine was recently honestly surprised that two government politicians got into trouble for personally taking money from mask-producing companies in return for securing government orders. He was not surprised that the politicians were corrupt. He was surprised that there were any repercussions. He thought that politicians could totally get away with this. How did he acquire that belief? A while earlier it was revealed that another politician received stock options from an IT startup, for which he then put in a good word at the government. He only got shit from the press, but his career is still going well.
My friend had previously thought that there was a strong norm against politicians being corrupt and that any violation of this norm would end their career. The politician who got away with the stock option affair was strong evidence against such a norm being enforced. Thus, he updated towards the belief that corrupt politicians can just get away with anything.
But how does this constitute a violation of rationality? Wasn’t I saying that this was some sort of fallacy? What did he do wrong?
His mistake was this: He assumed that, if there was a norm against corruption, it would follow a definition of corruption that did in some way actually corresponds to the degree to which the act was harmful. Let’s look at this in a graphical way:
Norms here are represented with lines, separating the forbidden acts on the left of the line from the allowed acts on the right of the line. His hypothesis space only consisted of straight lines. His prior was that there was a norm enforced (norm A). Then he observed act 1 occur unpunished. This made him update that there was either no norm or maybe a norm as far out as norm B. Then he observed act 2 being punished and was confused. The true norm that was enforced all along was norm C, which looks pretty gerrymandered. Does it make sense that MPs are obliged to formally record monetary payments they receive, but not stock options? Not really, but that is how it is.
I have also seen the same play out in other contexts. For example, after the FSB (probably) killed a former Chechen War commander in Berlin, my roommate thought that the Russian secret service would just kill anyone they held a grudge against and expected a journalist might be their next target after making fun of Putin. I think that expectation was totally unrealistic. It stems from the assumption that all lives should be protected equally. But in reality, if Putin kills a rebellious war commander or a double agent, other countries react with consequences that he is willing to take. But if he started killing western journalists, NATO would probably get really mad.
I think as a rule of thumb, we should expect gerrymandered lines in the sand when powerful people try to get around some norm.
So, next time, you see some powerful person get away with something you didn’t expect them to, consider that there is a line in the sand, just not where you think it is.
There is a line in the sand, just not where you think it is
There is a mistake in updating beliefs around norms that I have seen multiple times in different contexts made by different people. This is my attempt to classify this (fallacy? /mistake?), in order to give it a name and make it easier to refer to.
A friend of mine was recently honestly surprised that two government politicians got into trouble for personally taking money from mask-producing companies in return for securing government orders. He was not surprised that the politicians were corrupt. He was surprised that there were any repercussions. He thought that politicians could totally get away with this. How did he acquire that belief? A while earlier it was revealed that another politician received stock options from an IT startup, for which he then put in a good word at the government. He only got shit from the press, but his career is still going well.
My friend had previously thought that there was a strong norm against politicians being corrupt and that any violation of this norm would end their career. The politician who got away with the stock option affair was strong evidence against such a norm being enforced. Thus, he updated towards the belief that corrupt politicians can just get away with anything.
But how does this constitute a violation of rationality? Wasn’t I saying that this was some sort of fallacy? What did he do wrong?
His mistake was this: He assumed that, if there was a norm against corruption, it would follow a definition of corruption that did in some way actually corresponds to the degree to which the act was harmful. Let’s look at this in a graphical way:
Norms here are represented with lines, separating the forbidden acts on the left of the line from the allowed acts on the right of the line. His hypothesis space only consisted of straight lines. His prior was that there was a norm enforced (norm A). Then he observed act 1 occur unpunished. This made him update that there was either no norm or maybe a norm as far out as norm B. Then he observed act 2 being punished and was confused. The true norm that was enforced all along was norm C, which looks pretty gerrymandered. Does it make sense that MPs are obliged to formally record monetary payments they receive, but not stock options? Not really, but that is how it is.
I have also seen the same play out in other contexts. For example, after the FSB (probably) killed a former Chechen War commander in Berlin, my roommate thought that the Russian secret service would just kill anyone they held a grudge against and expected a journalist might be their next target after making fun of Putin. I think that expectation was totally unrealistic. It stems from the assumption that all lives should be protected equally. But in reality, if Putin kills a rebellious war commander or a double agent, other countries react with consequences that he is willing to take. But if he started killing western journalists, NATO would probably get really mad.
I think as a rule of thumb, we should expect gerrymandered lines in the sand when powerful people try to get around some norm.
So, next time, you see some powerful person get away with something you didn’t expect them to, consider that there is a line in the sand, just not where you think it is.