It’s an interesting pattern, and the examples are fun to think about. I can produce explanations for a few categories. A very general statement, covering most of your examples, is, “To a naive person, people who don’t share your goals or your background may react to your policies in ways you didn’t expect or even imagine”, which is downstream of the typical mind fallacy and other common mistakes.
A subcategory is “If people really want to do a thing (buy some good or service at market price; know an interesting fact; avoid doing something unpleasant; etc.), then any attempted suppression creates strong incentives to work around it, which often has fascinating results”. As shminux mentions, perverse incentives can happen, and the Streisand effect actually goes somewhat beyond it (there I would say your intervention hits a preexisting immune system—people try to avoid being tricked, and either instinct or habit tells them that someone trying to hide a fact from you is a sign they’re trying to trick you).
Another very general explanation: “If a complex system is already somewhat optimized in one direction, then there are a lot more ways to make it worse than to make it better, and therefore our priors on a naive attempt to ‘improve’ it should be pretty bad.” Chesterton’s fence is related.
That said, I would definitely not call this “enantiodromia” a “principle”—to me that implies it’s a scientific or mathematical fact, and furthermore suggests that there’s some mysterious force that specifically causes it. No: There are mundane explanations for everything, and there are heuristics you can learn (and should learn, especially if your career has any chance of affecting serious policy decisions), probably most importantly “Having thought of your proposal, try imagining the ways the different involved parties might react, and imagine ways it could go wrong”. There are forces opposing the proposals, but they’re different in each case (the people who just want good lawyers; the people who abuse celibacy norms; the virus researchers), and there is no mysterious force causing all of them (the thing that seems most like it is the “negative prior on naive policy proposals”).
It reminds me of the concept of “synchronicity”, the idea that you see more coincidences than appear to be explained by causality or by chance. As far as I can tell, the explanation for that is simply (a) sometimes the coincidence is explained by facts you’re unaware of (e.g. three friends happen to mention a specific unusual topic during a week, and you don’t know there was an article recently published on that topic, and all of them read it, or perhaps talked to someone who read it); (b) selective memory and reporting: a thousand things happen to you, most of which are common non-coincidences, and you remember only the freak coincidence.
If “synchronicity” is a fact, it’s a fact about your imperfect mind—that you don’t know the true causal connection and your memory is biased—not a fact about the world; I don’t think there’s some mysterious force deliberately throwing coincidences at you. But it seems that the guy who came up with the concept, Carl Jung, was (uncharitable explanation alert) so epistemically arrogant that he believed it was a mysterious fact about the world rather than about his imperfect mind.
Synchronicity (German: Synchronizität) is a concept first introduced by analytical psychologist Carl G. Jung “to describe circumstances that appear meaningfully related yet lack a causal connection.”[1] In contemporary research, synchronicity experiences refer to one’s subjective experience that coincidences between events in one’s mind and the outside world may be causally unrelated to each other yet have some other unknown connection.[2] [...]
Jung developed the theory of synchronicity as a hypothetical noncausal principle serving as the intersubjective or philosophically objective connection between these seemingly meaningful coincidences.[1][5] Mainstream science generally regards that any such hypothetical principle either does not exist or would not fall within the bounds of science.[6][7] [...]
Jung used the concept of synchronicity in arguing for the existence of the paranormal.[22] This idea was similarly explored by writer Arthur Koestler in his 1972 work The Roots of Coincidence[23] and was also taken up by the New Age movement.[6]
For this “enantiodromia”, I would again say: It’s a fact about the minds of those who make policies and were too unimaginative to think how they might go wrong, too incompetent to figure out how they would go wrong, or too careless to even try; not a fact about the world that there’s some mysterious force that enjoys dramatic irony and tries to make your policies go wrong. Only some epistemically arrogant person would think it’s a fact about the world… hey, guess what, it’s Carl Jung again! I seriously did make the connection to “synchronicity” before I looked up the Wikipedia on enantiodromia. Behold:
Enantiodromia (Ancient Greek: ἐνάντιος, romanized: enantios – opposite and δρόμος, dromos – running course) is a principle introduced in the West by psychiatrist Carl Jung. In Psychological Types, Jung defines enantiodromia as “the emergence of the unconscious opposite in the course of time.”[1] It is similar to the principle of equilibrium in the natural world, in that any extreme is opposed by the system in order to restore balance. When things get to their extreme, they turn into their opposite. Jung adds that “this characteristic phenomenon practically always occurs when an extreme, one-sided tendency dominates conscious life; in time an equally powerful counterposition is built up which first inhibits the conscious performance and subsequently breaks through the conscious control.”[1]
However, in Jungian terms, a thing psychically transmogrifies into its shadow opposite, in the repression of psychic forces that are thereby cathected into something powerful and threatening.
I see one advantage to talking about enantiodromia as a “principle”: diplomacy. If people care about status, it may be easier to tell a would-be reformer “Have you considered that your proposal might hit this mysterious abstract principle?” than “Have you considered that you may be too naive, unimaginative, and/or incompetent to see how your proposal will go wrong?”. (Note that I mean “incompetent” for the task, not compared to average; there may be policy failures that even the most competent person on Earth couldn’t foresee.) Indeed, as I see these references to earlier discussion of the “principle” during history, I suspect some of them are cases where people expected to punished if they openly criticized policy, so they talked about an abstraction instead; I even suspect something similar might be upstream of this thread. (I doubt it would actually work as a tactic, to persuade those who would respond badly to regular criticism; but perhaps people tried it.)
But there are other ways to address the politeness aspect. You can say things like “For any important proposal, we should try to imagine how it could go wrong”, mention perverse incentives if they seem relevant, mention the fun historical examples of screwups, and so on. I don’t think I’m an expert at politeness, but I think it’s a solvable problem. In any case, Less Wrong is not a place where people would talk about a supernatural force because it’s easier to be polite that way.
I see one advantage to talking about enantiodromia as a “principle”: diplomacy. [...] In any case, Less Wrong is not a place where people would talk about a supernatural force because it’s easier to be polite that way.
LessWrong is a mix of a place where some people have nothing at stake and other people have a lot at stake.
There are conversation where important information could plausibly exchanged but those conversations don’t happen because of various pressues. I do consider it useful to think how more of those conversations could happen. People are willing to share some information in private that they are not willing to publically share and narrowing that gap is useful.
It’s an interesting pattern, and the examples are fun to think about. I can produce explanations for a few categories. A very general statement, covering most of your examples, is, “To a naive person, people who don’t share your goals or your background may react to your policies in ways you didn’t expect or even imagine”, which is downstream of the typical mind fallacy and other common mistakes.
A subcategory is “If people really want to do a thing (buy some good or service at market price; know an interesting fact; avoid doing something unpleasant; etc.), then any attempted suppression creates strong incentives to work around it, which often has fascinating results”. As shminux mentions, perverse incentives can happen, and the Streisand effect actually goes somewhat beyond it (there I would say your intervention hits a preexisting immune system—people try to avoid being tricked, and either instinct or habit tells them that someone trying to hide a fact from you is a sign they’re trying to trick you).
Another very general explanation: “If a complex system is already somewhat optimized in one direction, then there are a lot more ways to make it worse than to make it better, and therefore our priors on a naive attempt to ‘improve’ it should be pretty bad.” Chesterton’s fence is related.
That said, I would definitely not call this “enantiodromia” a “principle”—to me that implies it’s a scientific or mathematical fact, and furthermore suggests that there’s some mysterious force that specifically causes it. No: There are mundane explanations for everything, and there are heuristics you can learn (and should learn, especially if your career has any chance of affecting serious policy decisions), probably most importantly “Having thought of your proposal, try imagining the ways the different involved parties might react, and imagine ways it could go wrong”. There are forces opposing the proposals, but they’re different in each case (the people who just want good lawyers; the people who abuse celibacy norms; the virus researchers), and there is no mysterious force causing all of them (the thing that seems most like it is the “negative prior on naive policy proposals”).
It reminds me of the concept of “synchronicity”, the idea that you see more coincidences than appear to be explained by causality or by chance. As far as I can tell, the explanation for that is simply (a) sometimes the coincidence is explained by facts you’re unaware of (e.g. three friends happen to mention a specific unusual topic during a week, and you don’t know there was an article recently published on that topic, and all of them read it, or perhaps talked to someone who read it); (b) selective memory and reporting: a thousand things happen to you, most of which are common non-coincidences, and you remember only the freak coincidence.
If “synchronicity” is a fact, it’s a fact about your imperfect mind—that you don’t know the true causal connection and your memory is biased—not a fact about the world; I don’t think there’s some mysterious force deliberately throwing coincidences at you. But it seems that the guy who came up with the concept, Carl Jung, was (uncharitable explanation alert) so epistemically arrogant that he believed it was a mysterious fact about the world rather than about his imperfect mind.
Am I being uncharitable? Judge for yourself:
For this “enantiodromia”, I would again say: It’s a fact about the minds of those who make policies and were too unimaginative to think how they might go wrong, too incompetent to figure out how they would go wrong, or too careless to even try; not a fact about the world that there’s some mysterious force that enjoys dramatic irony and tries to make your policies go wrong. Only some epistemically arrogant person would think it’s a fact about the world… hey, guess what, it’s Carl Jung again! I seriously did make the connection to “synchronicity” before I looked up the Wikipedia on enantiodromia. Behold:
Right. Well. I will just say that “people figuring out that a bounty on dead cobras incentivized them to breed cobras” does not require any invocation of unconscious opposites or psychic transmogrification to explain.
I see one advantage to talking about enantiodromia as a “principle”: diplomacy. If people care about status, it may be easier to tell a would-be reformer “Have you considered that your proposal might hit this mysterious abstract principle?” than “Have you considered that you may be too naive, unimaginative, and/or incompetent to see how your proposal will go wrong?”. (Note that I mean “incompetent” for the task, not compared to average; there may be policy failures that even the most competent person on Earth couldn’t foresee.) Indeed, as I see these references to earlier discussion of the “principle” during history, I suspect some of them are cases where people expected to punished if they openly criticized policy, so they talked about an abstraction instead; I even suspect something similar might be upstream of this thread. (I doubt it would actually work as a tactic, to persuade those who would respond badly to regular criticism; but perhaps people tried it.)
But there are other ways to address the politeness aspect. You can say things like “For any important proposal, we should try to imagine how it could go wrong”, mention perverse incentives if they seem relevant, mention the fun historical examples of screwups, and so on. I don’t think I’m an expert at politeness, but I think it’s a solvable problem. In any case, Less Wrong is not a place where people would talk about a supernatural force because it’s easier to be polite that way.
LessWrong is a mix of a place where some people have nothing at stake and other people have a lot at stake.
There are conversation where important information could plausibly exchanged but those conversations don’t happen because of various pressues. I do consider it useful to think how more of those conversations could happen. People are willing to share some information in private that they are not willing to publically share and narrowing that gap is useful.