IMO indirect effects and leverage are the most important factors here.
Almost all actions are taken habitually, rather as the result of bespoke strategic consideration, but what gets to be habitual is downstream of morality and material incentives. And morality exercises leverage via:
1) Reputational effects/RLHF (it’s “cheap” to judge your neighbor and expensive to walk the walk yourself, but many many neighbors judging each other differently produces different habit regimes)
2) Acausal trade once there’s common knowledge that the trade exists
3) Consciously reworking incentives systems (if you keep other people as slaves we’ll chop your head off, etc)
Back when I was a more orthodox marxist, I thought that material incentives were downstream of technological regimes, and that morality tended to be downstream of the incentives, such that morality tended to be lower-leverage even if people took plenty of actions for moral reasons. I still think all those effects are real; I’m just more of a moral realist now so I don’t think morality is as pliable as all that—it’s downstream of True Morality and higher-leverage.
There’s an “equilibrium disequilibrium” situation where everyone can see that everyone benefits from everyone doing X, and you can defect and reach high rewards from Y, where individuals doing X vs Y is hard to observe directly, and so there are periodic cycles of moralized attempts to get to a higher X-based equilibrium and people tearing through the commons by Y-ing (and becoming objects of emulation since many other, perfectly good, things could have led to their success.)
This is all in principle orthogonal to whether morality is harming or helping—I’d expect the same incentives when morality is being harmful. But (1) on moral realism here I think there’s an inherent bias towards being helpful rather than harmful that is just a function of “intentional actions have some kind of relation to what they’re intending at all,” if you want to throw this out then you basically take out the idea that there are people acting rather than just behaving, (2) most harm from moral action is either in jumpstarting preference cascade bubbles that naturally collapse pretty quickly, or in periodic (literal or metaphorical) vigilante violence that itself would be impossible to defend against without all the morality-based stuff above.
In the future decisive actions could make morality have been net-negative—one could imagine a future where the desire to punish at a crucial juncture created permanent hells, such that it would be better for the galaxy to have been converted to hedonium or some even less worthwhile goo. This is an instance of the broader principle that a process biased towards positive x can produce negative x with small sample sizes.
IMO indirect effects and leverage are the most important factors here.
Almost all actions are taken habitually, rather as the result of bespoke strategic consideration, but what gets to be habitual is downstream of morality and material incentives. And morality exercises leverage via:
1) Reputational effects/RLHF (it’s “cheap” to judge your neighbor and expensive to walk the walk yourself, but many many neighbors judging each other differently produces different habit regimes)
2) Acausal trade once there’s common knowledge that the trade exists
3) Consciously reworking incentives systems (if you keep other people as slaves we’ll chop your head off, etc)
Back when I was a more orthodox marxist, I thought that material incentives were downstream of technological regimes, and that morality tended to be downstream of the incentives, such that morality tended to be lower-leverage even if people took plenty of actions for moral reasons. I still think all those effects are real; I’m just more of a moral realist now so I don’t think morality is as pliable as all that—it’s downstream of True Morality and higher-leverage.
There’s an “equilibrium disequilibrium” situation where everyone can see that everyone benefits from everyone doing X, and you can defect and reach high rewards from Y, where individuals doing X vs Y is hard to observe directly, and so there are periodic cycles of moralized attempts to get to a higher X-based equilibrium and people tearing through the commons by Y-ing (and becoming objects of emulation since many other, perfectly good, things could have led to their success.)
This is all in principle orthogonal to whether morality is harming or helping—I’d expect the same incentives when morality is being harmful. But (1) on moral realism here I think there’s an inherent bias towards being helpful rather than harmful that is just a function of “intentional actions have some kind of relation to what they’re intending at all,” if you want to throw this out then you basically take out the idea that there are people acting rather than just behaving, (2) most harm from moral action is either in jumpstarting preference cascade bubbles that naturally collapse pretty quickly, or in periodic (literal or metaphorical) vigilante violence that itself would be impossible to defend against without all the morality-based stuff above.
In the future decisive actions could make morality have been net-negative—one could imagine a future where the desire to punish at a crucial juncture created permanent hells, such that it would be better for the galaxy to have been converted to hedonium or some even less worthwhile goo. This is an instance of the broader principle that a process biased towards positive x can produce negative x with small sample sizes.