The obvious reason that Moloch is the enemy is that it destroys everything we value in the name of competition and survival. But this is missing the bigger picture. We value what we value because, in our ancestral environment, those tended to be the things that helped us with competition and survival. If the things that help us compete and survive end up changing, then evolution will ensure that the things we value change as well.
To borrow a metaphor: Elua cheats. The hedonic treadmill has nothing on the axiological treadmill.
Consider a thought experiment. In Meditations on Moloch, Scott Alexander dreams up a dictatorless dystopia:
Imagine a country with two rules: first, every person must spend eight hours a day giving themselves strong electric shocks. Second, if anyone fails to follow a rule (including this one), or speaks out against it, or fails to enforce it, all citizens must unite to kill that person. Suppose these rules were well-enough established by tradition that everyone expected them to be enforced.
So you shock yourself for eight hours a day, because you know if you don’t everyone else will kill you, because if they don’t, everyone else will kill them, and so on. Every single citizen hates the system, but for lack of a good coordination mechanism it endures. From a god’s-eye-view, we can optimize the system to “everyone agrees to stop doing this at once”, but no one within the system is able to effect the transition without great risk to themselves.
Even if this system came into being ex nihilo it probably wouldn’t be stable in reality; a population that spends eight hours a day receiving strong shocks isn’t going to be able to feed itself, or reproduce. But assume for a moment that this system starts out economically and biologically stable (that is, people can still eat, and reproduce at the rate of replacement, despite the electric shocks, and that there are no outside countries ready to invade). What do we expect to happen over the long run?
Well, obviously there’s a strong evolutionary pressure to be tolerant to electric shocks. People who can tolerate those shocks better will do better on average than those who can’t. However, there’s another more subtle pressure at play: the pressure to ensure you shock yourself. After all, if you forget to shock yourself, or choose not to, then you are immediately killed. So the people in this country will slowly evolve reward and motivational systems such that, from the inside, it feels like they want to shock themselves, in the same way (though maybe not to the same degree) that they want to eat. Shocking themselves every day becomes an intrinsic value to them. Eventually, it’s no longer a dystopia at all.
They would be aghast at a society like ours, where Moloch has destroyed the value of receiving electrical shocks, all in the name of more perfect competition.
The Axiological Treadmill
The obvious reason that Moloch is the enemy is that it destroys everything we value in the name of competition and survival. But this is missing the bigger picture. We value what we value because, in our ancestral environment, those tended to be the things that helped us with competition and survival. If the things that help us compete and survive end up changing, then evolution will ensure that the things we value change as well.
To borrow a metaphor: Elua cheats. The hedonic treadmill has nothing on the axiological treadmill.
Consider a thought experiment. In Meditations on Moloch, Scott Alexander dreams up a dictatorless dystopia:
Even if this system came into being ex nihilo it probably wouldn’t be stable in reality; a population that spends eight hours a day receiving strong shocks isn’t going to be able to feed itself, or reproduce. But assume for a moment that this system starts out economically and biologically stable (that is, people can still eat, and reproduce at the rate of replacement, despite the electric shocks, and that there are no outside countries ready to invade). What do we expect to happen over the long run?
Well, obviously there’s a strong evolutionary pressure to be tolerant to electric shocks. People who can tolerate those shocks better will do better on average than those who can’t. However, there’s another more subtle pressure at play: the pressure to ensure you shock yourself. After all, if you forget to shock yourself, or choose not to, then you are immediately killed. So the people in this country will slowly evolve reward and motivational systems such that, from the inside, it feels like they want to shock themselves, in the same way (though maybe not to the same degree) that they want to eat. Shocking themselves every day becomes an intrinsic value to them. Eventually, it’s no longer a dystopia at all.
They would be aghast at a society like ours, where Moloch has destroyed the value of receiving electrical shocks, all in the name of more perfect competition.
[Cross-posted from Grand, Unified, Empty.]