The questions you list seem sort of generated by “what causes good things?”. I’m not sure if this is what you mean but it suggests an engineering mindset to me. (i.e. lets understand how culture and civilization works so we can make it be better (by our current lights? or current meta-lights?)
I’m currently looking at this from the generator that output the Coordination Frontier sequence, and I think I’m looking for something like “what are good principles for handling defection in weird confusing circumstances”, in ways that I’d expect aliens to independently converge on.
(I think the set of things we cluster around the word ‘morality’ includes both “coordination” as well as “doing whatever it is you value that’s worth coordinating for”. I am talking here primarily about “coordination”, but the two things aren’t crisp enough in my head to fully distinguish them)
Rambly thoughts that’ll maybe become a blogpost later:
Say you have two agents that meet in the wilderness. They have different values. Maybe one wants to make Art and Babies, and one wants to make PaperClips. Or maybe they both want Art and Babies but they want their particular Art and Babies.
They want the same resources.
In this example let’s say they speak the same language.
I think if one agent is vastly more powerful than the other, and neither agent is some kind of evolved values for empathy/universal-altruism, then I think it’s overdetermined that Alice just stomps all over Bob and takes his stuff.
If Alice and Bob are comparably powerful, such that it’s not immediately obvious who would win, then they do some complicated negotiation. The negotiation scheme depends on local features of their respective cultures, and the environment. (The ‘negotiation’ might take the form of ‘go to war’, but that’s costly and hopefully there are other less costly options).
But my guess is that there are at least some schelling norms that aliens would independently discover. (possible examples: honesty, not shooting messengers, keeping promises). They might each wish to strategically defect from those norms, but agree that they’re overall better off if they can bootstrap an enforcement system that changes the payoff for defection.
I think the longer an agent/culture has had time to think, the more complicated norms they will have time to discover.
This is all background for a half-formed thoughts that feel relevant to this post:
1) I suspect that “Just Punishment” is a schelling norm. i.e. it’s okay to punish defectors. This can take different forms like “it’s okay to kill in self defense”, “it’s okay to use violence to defend against violence”, etc
2) I suspect that “being in good or bad standing, and having to ‘pay for damages done’ to get back into good standing”, is a schelling norm.
3) I think “figure out how to bootstrap to actual trust/honesty, so you can save a ton of cognitive effort that you’d spend lying, or checking if other people are lying”, is a schelling norm. (Where the bootstrapping might include periodic surprise inspections that punish you massively if you turn out to be lying)
4) I think “figure out what deal you wish you had made, if you end up in a situation where you didn’t think to make a deal in advance, and then act as if you had made that deal”, is a good norm in many situations. I think it’s easier among people who trust each other. I think among people who don’t trust each other it can still work fairly well in some cases. (This is a bit handwavy at this point and probably needs a blogpost to flesh out)
i.e. if Alice and Bob get into a big protracted fight and end up in weird blood feud, they might say “okay, wait, wait, hold up. Man, this sucks. We didn’t have any clear coordination principles or communication frames when we got into this mess. What do we wish we had done instead of getting into a stupid blood feud?”. Where the answer isn’t “hold hands and sing kumbaya”, but might be “we wish we’d taken stock of our actual negotiating positions and resolved the dispute in some less costly way..” (this isn’t a great example, I’ll see if I can come up with a better one later)
5) points 1-4 apply differently to small groups who all know each other, vs large nations with many actors who vaguely share cultures. Nations are sort of like agents, and sort of not. I’m a bit confused how to think about that. (See that part of Hamilton where Jefferson says “we should honor the commitment we made to the French. What would Lafayette think of you turning your back on them?” and Hamilton replying with “the actual people we made the deal with are all dead”)
6) “what deal do you wish you had made?” is a bit confusing when you’ve gained new principles (coordination-wise, or otherwise) in the meanwhile
7) in many cases, the cognitive effort of figuring out what “should” have happened is too complicated and costly and you have to just sort of declare bankrupcty on the whole thing. But, if you do that a lot you give up on certain types of trust that things will ever be “made right”, which limits what sort of deals you can make in the future.
...
...tying this back to the OP is still a bit confusing. Stopping there for now.
Okay. So to try to tie this better back to the OP.
I want to reconcile the fact that my “common sense” morality includes “don’t steal from people, even the outgroup”, but “seems like empires are basically built out of theft (among other things) and like they were a key piece of how I even got my morality in the first place.”
And I think I was okay with “in practice, the way I got my morality was messy and build on atrocities, but at least I have a sense of how to do things going forward.” But the notion that “in principle, it may not have even been possible to get my morality without it being built on atrocities” is somehow deeply unsettling to me.
And maybe what I’m supposed to do is just grieve for that and move on with my day.
But I think not. Because I think we’re not necessarily done with empires needing to be built on (what I consider) immoral actions, for the greater good. (Or: it will probably turn out that at least some immoral empires will generate more goodness than could have been generated alternatively)
But it’s not like commonsense morality is meaningless. Theft is still bad for all the reasons it’s normally bad. I don’t want to stop punishing people for theft. Empires (which includes various large corporations) aren’t automatically for the greater good, so the rule of “you get punished for theft unless you’re too powerful” doesn’t actually solve anything even if my.
So the point of the previous comment was to grope towards “what norm do I actually want everyone to try to enforce, that has a shot at successfully applying to empires? Ideally that empire leaders should actually want to help enforce to create a higher-order incentive landscape?”
In a simpler ecosystem of individual agents, I think a reasonable principle is something like “it’s potentially okay for Alice to violate some norms to gain resources and accomplish a goal, if Alice later ‘makes good’ by paying some appropriate fine (ideally to the people who were harmed by her norm violation).”
But, this is much harder to apply to...
...vague-blobs of agents (who is useful to blame for crimes against the Irish? The current prime minister? The descendants of Oliver Cromwell? Whichever lower level official carried out various actions?)
...agents who are now dead (is there any way I can meaningfully punish Oliver Cromwell?)
...messy situations with lots of back and forth small crimes that escalated (My vague history sense is that England tried to stamp out Catholicism in Ireland, but I’m assuming the original Catholic settlers tried to stamp out other native practices, and everyone probably did random cattle raiding or conquest in various directions)
...
Thinking through this led me to some potential solutions:
It feels more robust to reward people who do good* than to punish people who do bad. It’s neither fair nor particularly possible to punish people who hadn’t been part of my moral framework, who’s deeds are mostly lost to the mists of time (and this is symmetrical for future people who might not share my framework but judge me by theirs)
Hmm. I guess I don’t necessarily want to punish people who did wrong, but I would ideally like to compensate people who were wronged.
A thing I can aim for for myself is to be the sort of person who tracks when I end up violating what I think should be a norm, and work to make it right in some way.
A thing I can also aim for myself is to work to ensure that organizations I participate in are robust, coherent organizations that are easier to audit and assign credit in, so I don’t contribute to a vague blob that’s hard to hold accountable.
There is something nice about the virtue of “ensuring that there is a record”, and being legible to historians. This enables future people, who might have a lot more resources, to allocate some kind of future rewards for people who put effort into making their organizations legible/coherent/agenty, and who attempted to do good with their imperial winnings.
(sufficiently advanced posthumans can solve this by literally implementing simulated ancestor heaven, and if that turns out to be intractable, you can instead do things like “figure out what someone
While thinking through all of this, one thing I found myself thinking of were things like slave reparations. Up until 10 minutes ago I was thinking something like “I think this has made me more pro-reparations”. The thought I had just now was “actually, the problem with reparations right now is that America/UK/whoever are too much of a vague messy blob to institute any morally complex norm reconciliation, and the primary thing to focus on if you’re worried about righting-past-wrongs is to help build a more coherent, agenty world that’s capable of doing anything on purpose at all.”
A thing this reminds me of is that Robert Moses (city planner for early-mid 20th century NYC) did a bunch of horrible destructive (and often racist) things in the service of, among other things, highways. And those things were horribly destructive (and often racist), but I kinda suspect that if he’d done them in the name of mass transit a lot of the people condemning him would be saying “man of his times, what can you do?” and “can’t make an omelette without breaking a few eggs”, unless people were unhappy with the results of more mass transit/fewer highways in this alternate NYC, at which point they’d go back to condemning him for all the things we condemn him for now.
I would rather Moses had built mass transit rather than highways in the 30s, but I don’t think it was obviously the wrong trade-off at the time. A bunch of things looked like good ideas in the 30s and didn’t pan out.
Nod.
The questions you list seem sort of generated by “what causes good things?”. I’m not sure if this is what you mean but it suggests an engineering mindset to me. (i.e. lets understand how culture and civilization works so we can make it be better (by our current lights? or current meta-lights?)
I’m currently looking at this from the generator that output the Coordination Frontier sequence, and I think I’m looking for something like “what are good principles for handling defection in weird confusing circumstances”, in ways that I’d expect aliens to independently converge on.
(I think the set of things we cluster around the word ‘morality’ includes both “coordination” as well as “doing whatever it is you value that’s worth coordinating for”. I am talking here primarily about “coordination”, but the two things aren’t crisp enough in my head to fully distinguish them)
Rambly thoughts that’ll maybe become a blogpost later:
Say you have two agents that meet in the wilderness. They have different values. Maybe one wants to make Art and Babies, and one wants to make PaperClips. Or maybe they both want Art and Babies but they want their particular Art and Babies.
They want the same resources.
In this example let’s say they speak the same language.
I think if one agent is vastly more powerful than the other, and neither agent is some kind of evolved values for empathy/universal-altruism, then I think it’s overdetermined that Alice just stomps all over Bob and takes his stuff.
If Alice and Bob are comparably powerful, such that it’s not immediately obvious who would win, then they do some complicated negotiation. The negotiation scheme depends on local features of their respective cultures, and the environment. (The ‘negotiation’ might take the form of ‘go to war’, but that’s costly and hopefully there are other less costly options).
But my guess is that there are at least some schelling norms that aliens would independently discover. (possible examples: honesty, not shooting messengers, keeping promises
). They might each wish to strategically defect from those norms, but agree that they’re overall better off if they can bootstrap an enforcement system that changes the payoff for defection.I think the longer an agent/culture has had time to think, the more complicated norms they will have time to discover.
This is all background for a half-formed thoughts that feel relevant to this post:
1) I suspect that “Just Punishment” is a schelling norm. i.e. it’s okay to punish defectors. This can take different forms like “it’s okay to kill in self defense”, “it’s okay to use violence to defend against violence”, etc
2) I suspect that “being in good or bad standing, and having to ‘pay for damages done’ to get back into good standing”, is a schelling norm.
3) I think “figure out how to bootstrap to actual trust/honesty, so you can save a ton of cognitive effort that you’d spend lying, or checking if other people are lying”, is a schelling norm. (Where the bootstrapping might include periodic surprise inspections that punish you massively if you turn out to be lying)
4) I think “figure out what deal you wish you had made, if you end up in a situation where you didn’t think to make a deal in advance, and then act as if you had made that deal”, is a good norm in many situations. I think it’s easier among people who trust each other. I think among people who don’t trust each other it can still work fairly well in some cases. (This is a bit handwavy at this point and probably needs a blogpost to flesh out)
i.e. if Alice and Bob get into a big protracted fight and end up in weird blood feud, they might say “okay, wait, wait, hold up. Man, this sucks. We didn’t have any clear coordination principles or communication frames when we got into this mess. What do we wish we had done instead of getting into a stupid blood feud?”. Where the answer isn’t “hold hands and sing kumbaya”, but might be “we wish we’d taken stock of our actual negotiating positions and resolved the dispute in some less costly way..” (this isn’t a great example, I’ll see if I can come up with a better one later)
5) points 1-4 apply differently to small groups who all know each other, vs large nations with many actors who vaguely share cultures. Nations are sort of like agents, and sort of not. I’m a bit confused how to think about that. (See that part of Hamilton where Jefferson says “we should honor the commitment we made to the French. What would Lafayette think of you turning your back on them?” and Hamilton replying with “the actual people we made the deal with are all dead”)
6) “what deal do you wish you had made?” is a bit confusing when you’ve gained new principles (coordination-wise, or otherwise) in the meanwhile
7) in many cases, the cognitive effort of figuring out what “should” have happened is too complicated and costly and you have to just sort of declare bankrupcty on the whole thing. But, if you do that a lot you give up on certain types of trust that things will ever be “made right”, which limits what sort of deals you can make in the future.
...
...tying this back to the OP is still a bit confusing. Stopping there for now.
Okay. So to try to tie this better back to the OP.
I want to reconcile the fact that my “common sense” morality includes “don’t steal from people, even the outgroup”, but “seems like empires are basically built out of theft (among other things) and like they were a key piece of how I even got my morality in the first place.”
And I think I was okay with “in practice, the way I got my morality was messy and build on atrocities, but at least I have a sense of how to do things going forward.” But the notion that “in principle, it may not have even been possible to get my morality without it being built on atrocities” is somehow deeply unsettling to me.
And maybe what I’m supposed to do is just grieve for that and move on with my day.
But I think not. Because I think we’re not necessarily done with empires needing to be built on (what I consider) immoral actions, for the greater good. (Or: it will probably turn out that at least some immoral empires will generate more goodness than could have been generated alternatively)
But it’s not like commonsense morality is meaningless. Theft is still bad for all the reasons it’s normally bad. I don’t want to stop punishing people for theft. Empires (which includes various large corporations) aren’t automatically for the greater good, so the rule of “you get punished for theft unless you’re too powerful” doesn’t actually solve anything even if my.
So the point of the previous comment was to grope towards “what norm do I actually want everyone to try to enforce, that has a shot at successfully applying to empires? Ideally that empire leaders should actually want to help enforce to create a higher-order incentive landscape?”
In a simpler ecosystem of individual agents, I think a reasonable principle is something like “it’s potentially okay for Alice to violate some norms to gain resources and accomplish a goal, if Alice later ‘makes good’ by paying some appropriate fine (ideally to the people who were harmed by her norm violation).”
But, this is much harder to apply to...
...vague-blobs of agents (who is useful to blame for crimes against the Irish? The current prime minister? The descendants of Oliver Cromwell? Whichever lower level official carried out various actions?)
...agents who are now dead (is there any way I can meaningfully punish Oliver Cromwell?)
...messy situations with lots of back and forth small crimes that escalated (My vague history sense is that England tried to stamp out Catholicism in Ireland, but I’m assuming the original Catholic settlers tried to stamp out other native practices, and everyone probably did random cattle raiding or conquest in various directions)
...
Thinking through this led me to some potential solutions:
It feels more robust to reward people who do good* than to punish people who do bad. It’s neither fair nor particularly possible to punish people who hadn’t been part of my moral framework, who’s deeds are mostly lost to the mists of time (and this is symmetrical for future people who might not share my framework but judge me by theirs)
Hmm. I guess I don’t necessarily want to punish people who did wrong, but I would ideally like to compensate people who were wronged.
A thing I can aim for for myself is to be the sort of person who tracks when I end up violating what I think should be a norm, and work to make it right in some way.
A thing I can also aim for myself is to work to ensure that organizations I participate in are robust, coherent organizations that are easier to audit and assign credit in, so I don’t contribute to a vague blob that’s hard to hold accountable.
There is something nice about the virtue of “ensuring that there is a record”, and being legible to historians. This enables future people, who might have a lot more resources, to allocate some kind of future rewards for people who put effort into making their organizations legible/coherent/agenty, and who attempted to do good with their imperial winnings.
(sufficiently advanced posthumans can solve this by literally implementing simulated ancestor heaven, and if that turns out to be intractable, you can instead do things like “figure out what someone
While thinking through all of this, one thing I found myself thinking of were things like slave reparations. Up until 10 minutes ago I was thinking something like “I think this has made me more pro-reparations”. The thought I had just now was “actually, the problem with reparations right now is that America/UK/whoever are too much of a vague messy blob to institute any morally complex norm reconciliation, and the primary thing to focus on if you’re worried about righting-past-wrongs is to help build a more coherent, agenty world that’s capable of doing anything on purpose at all.”
A thing this reminds me of is that Robert Moses (city planner for early-mid 20th century NYC) did a bunch of horrible destructive (and often racist) things in the service of, among other things, highways. And those things were horribly destructive (and often racist), but I kinda suspect that if he’d done them in the name of mass transit a lot of the people condemning him would be saying “man of his times, what can you do?” and “can’t make an omelette without breaking a few eggs”, unless people were unhappy with the results of more mass transit/fewer highways in this alternate NYC, at which point they’d go back to condemning him for all the things we condemn him for now.
I would rather Moses had built mass transit rather than highways in the 30s, but I don’t think it was obviously the wrong trade-off at the time. A bunch of things looked like good ideas in the 30s and didn’t pan out.
[Source: primarily The Power Broker].