Because the length of Scott’s Moloch post greatly exceeds my working memory (to the extent that I had trouble remembering what the point was by the end) I made these notes. I hope this is the right place to share them.
Notes on Moloch (ancient god of child sacrifice)
http://slatestarcodex.com/2014/07/30/meditations-on-moloch/
Intro—no real content.
Moloch as coordination failure: everyone makes a sacrifice to optimize for a zero-sum competition, ends up with the same relative status, but worse absolute status.
10 examples: Prisoner’s Dilemma, dollar auctions, fish-farming story (tragedy of the commons), Malthusian trap, ruthless/exploitative Capitalist markets, the two-income trap, agriculture, arms races, cancer, political race to the bottom (lowering taxes to attract business)
4 partial examples: inefficient education, inefficient science, government corruption (corporate welfare), Congress (representatives voting against good of nation for good of constituency)
Existing systems are created by incentive structures, not agents, e.g. Las Vegas caused by a known bias in human reward circuitry, not optimization for human values.
But sometimes we move uphill anyway. Possible explanations:
Excess resources / we are in the dream time and can afford non-competitive behavior.
Physical limitations to what can be sacrificed
Economic competition actually producing positive utility for consumers (but this is fragile)
Coordination, e.g. via governments, guilds, friendships, etc.
Technology/ingenuity creates new opportunities to fall into such traps. Technology overcomes physical limitations, consumes excess resources. Automation further decouples economic activity from human values. Technology can improve coordination, but can also exacerbate existing conflicts by giving all sides more power.
AGI opens up whole new worlds of traps: Yudkowsky’s paperclipper, Hanson’s subsistence-level ems, Bostrom’s Disneyland with no children.
6 & 7. Gnon—basically the god of the conservative scarcity mindset. Nick Land advocates compliance; Nyan wants to capture Gnon and build a walled garden. Scott warns that Moloch is far more terrifying than Gnon and will kill both of them anyway.
8 & 9. So we have to kill this Moloch guy, by lifting a better God to Heaven (Elua).
From Kahneman’s Thinking, Fast and Slow (p 325):
Do you (r_claypool) have reason to suspect that Christianity is much more likely to be true than other, (almost-) mutually exclusive supernatural worldviews like, say, Old Norse Paganism? If not, then 5% for Christianity is absurdly high.