That makes sense, but something feels off about it.
It seems to assume that all agents have the same payoff matrix, that the metrics are objective rather than subjective, that objective outcomes are the same as subjective outcomes, that the agent is not deceived about the payoff, that agents optimize for the same thing, that all agents have access to the same strategies.
It’s also a mistake to compare biological behaviour to rational optimization. If I’m hungry, I gather food, and once I have enough food, I stop gathering food. The value of food depends on its scarcity, and any needs I have can be sated, limiting the destruction on my environment. Animals don’t generally engage in unbounded behaviour. The seven deadly sins are a pathological failure in which one seeks X when they can only be satisfied by Y. X gives a reward which feels like Y, but only as a short-lived substanceless imitation. What religious people conceptionalize as evil is behaviour which aligns with this pathology. Evil seems to be insatiability itself, thought it’s often described in terms of its consequences.
The story of “Goddess of Cancer vs the Goddess of Everything Else” seems wrong to me. I don’t think there was any molochian behaviour before human beings. Ants do not destroy themselves. and I think it was limited up until the modern world because we weren’t intelligent, materialistic, and pathological enough for it to become a bigger issue than culture could protect against. Also, some cultures succeeded more than others. Socities successful in this manner are now called “high trust socities”.
There’s a book called “The future of the commons” which deal with social dilemmas. One quote goes “Even relatively large-scale collective-action problems are more likely to be solved when the necessary institutions are developed from below”. Taleb seems to have written Skin in the Game with a similar intuition for the conflict between emergent self-regulation and external regulation. Finally, Seeing like a state outlines a similar point, as does Ribbonfarms article on Legibility. What do all these have in common? they criticize modern aspects of the world, which appeared around the same time that Molochian problems started to get the upper hand.
Of course Moran processes are molochian—all pure replicators are. Just like grey goo, and cancer. The only non-molochian life is biological, sentient and conscious. This is exactly the set of life which can experience qualia, and the set of life which can be subjective, illogical, moral, and resist taking optimal actions.
That makes sense, but something feels off about it.
It seems to assume that all agents have the same payoff matrix, that the metrics are objective rather than subjective, that objective outcomes are the same as subjective outcomes, that the agent is not deceived about the payoff, that agents optimize for the same thing, that all agents have access to the same strategies.
It’s also a mistake to compare biological behaviour to rational optimization. If I’m hungry, I gather food, and once I have enough food, I stop gathering food. The value of food depends on its scarcity, and any needs I have can be sated, limiting the destruction on my environment. Animals don’t generally engage in unbounded behaviour. The seven deadly sins are a pathological failure in which one seeks X when they can only be satisfied by Y. X gives a reward which feels like Y, but only as a short-lived substanceless imitation. What religious people conceptionalize as evil is behaviour which aligns with this pathology. Evil seems to be insatiability itself, thought it’s often described in terms of its consequences.
The story of “Goddess of Cancer vs the Goddess of Everything Else” seems wrong to me. I don’t think there was any molochian behaviour before human beings. Ants do not destroy themselves. and I think it was limited up until the modern world because we weren’t intelligent, materialistic, and pathological enough for it to become a bigger issue than culture could protect against. Also, some cultures succeeded more than others. Socities successful in this manner are now called “high trust socities”.
There’s a book called “The future of the commons” which deal with social dilemmas. One quote goes “Even relatively large-scale collective-action problems are more likely to be solved when the necessary institutions are developed from below”. Taleb seems to have written Skin in the Game with a similar intuition for the conflict between emergent self-regulation and external regulation. Finally, Seeing like a state outlines a similar point, as does Ribbonfarms article on Legibility. What do all these have in common? they criticize modern aspects of the world, which appeared around the same time that Molochian problems started to get the upper hand.
Of course Moran processes are molochian—all pure replicators are. Just like grey goo, and cancer. The only non-molochian life is biological, sentient and conscious. This is exactly the set of life which can experience qualia, and the set of life which can be subjective, illogical, moral, and resist taking optimal actions.
Thoughts?