I’m not sure how relevant this line of questioning is to my main point.
Is your point trying to use the same definitions and ontology as in the post, and responding to a particular logical argument made within it?
If yes, perhaps it would help if you would pick one of the specific arguments you disagree with — say, the 5-part argument argument for “stealing is bad” as a cosmic Schelling norm — and identify what is the first statement or inference in it that you think is wrong.
If no, can you say what is your definition of “stealing” and “stealing is good”? The post gives a particular cosmically general definition of “stealing”, and also “is cosmically Schelling-good”, and a particular argument that stealing is cosmically Schelling-bad.
The post does not define “is good” — it only notes an asymmetry of encouragement effects between “is good” and “is bad” — so I’m not sure if you are intending to use the same concepts as the post to respond to its arguments, or if you want to redirect or broaden our focus to some other sense of what you mean by “stealing” and “coding agents” and “all the time”.
Is your point trying to use the same definitions and ontology as in the post, and responding to a particular logical argument made within it?
Yes to the former, no the latter. I was more trying to “sanity check” your overall approach against my intuitions and trying to explain why I feel very skeptical about it. But I can try to trace back what part of the post I start to disagree with:
Let’s talk about stealing as a concrete example, since we haven’t discussed that yet.
What’s the cosmic Schelling answer to the question, “Is stealing good or bad?”
I’m not sure if there’s an even earlier step, but here it jumps out at me that you seem to have chosen “stealing” as an example because it’s a highly salient moral question for humans, but it may well not be for other civilizations, like my hypothetical one. You seem to be implicitly assuming that everyone will try to converge on the same questions (because “cosmic Schelling question asks what answer beings would converge on when trying to converge on the same answer”, which seemingly wouldn’t make sense to do if others are not actually trying to converge on the same questions), whereas my intuition is that norms and coordination mechanisms in general may be highly context-dependent so this seems like an unjustified assumption.
Thanks for answering my questions, I am more oriented now!
here it jumps out at me that you seem to have chosen “stealing” as an example because it’s a highly salient moral question for humans
I don’t think that’s the case. On the contrary, here is the causal history of how I chose stealing:
A few years ago I was writing up some groundwork for a mathematical formalization of embedded agents, involving an information-theoretic boundary that distinguishes the agent from its environment… possibly a soft/fuzzy boundary, but a boundary nonetheless. While writing this, I noticed that wherever agents manage to persist over time, there are norms or ‘tendencies’ to respect their boundaries, almost tautologically. This lead me to write my Boundaries sequence.
I then looked for existing English words for what it means to (not) push or pull things through boundaries in various ways, and (not) stealing was the word for (not) taking stuff out of someone else’s boundary. Then, I would occasionally try to point out to people why this made stealing a relatively simple and thus convergently agreeable norm, but they didn’t seem sufficiently familiar with the dynamics of Schelling points for my point to be conveyed easily.
So I decided to write a post explaining how Schelling dynamics have a role to play in certain classes of meta-moral judgements, with an application at a cosmic scale, since at that scale people often have a lot of doubt and confusion about morality.
So, I don’t think I was looking to justify stealing, but rather, looking for word to refer to a fairly basic boundary-theoretic norm.
You seem to be implicitly assuming that everyone will try to converge on the same questions
No, this is quite carefully not assumed. Please see the section entitled “This essay is not very skimmable”, which is written to emphasize up-front the distinction between
thought experiment stipulations, versus
assertions about what large classes of real agents would say about those thought experiments.
The assumption of intentional convergence lives in the thought experiment stipulations, not assumptions about real agents. That is, the essay does not assume, as a belief about real-world agents, that they share an actual intention to converge on an answer. Although, some agents do have that intention, and such intentions might be made more prevalent as a result of the Schelling participation effect described in the post, in which case those intentions are a consequence of reasoning rather than an assumption about the reasoner.
Please see the section entitled “This essay is not very skimmable”
I had seen that warning, and was trying to keep track of the distinction, but apparently still failed. To check my understanding now:
cosmic Schelling answers are hypothetical answers in thought experiments where we assume that everyone is trying to converge to the same answers on the same questions
cosmic Schelling norms are just a subset of cosmic Schelling answers (“to a pro tanto moral question”), and therefore not necessarily actual norms in the dictionary sense of “a principle of right action binding upon the members of a group and serving to guide, control, or regulate proper and acceptable behavior”
In other words, the cosmic Schelling norm of an arbitrary pro tanto moral question probably exists in platonic space, but in most cases this would not be an actual norm in reality because (among other potential reasons) most beings in the cosmos would not actually be trying to converge on this particular question. Is this correct?
(If so, I’m confused how this usage of “norm” squares with your position as a compositional language realist, since compositionally it seems like a statement of the form “X is a cosmic Schelling norm” should imply that X is a norm?)
cosmic Schelling answers are hypothetical answers in thought experiments where we assume that everyone is trying to converge to the same answers on the same questions
Not quite. As I intend it: cosmic Schelling answers are real answers to real questions about hypothetical scenarios in which everyone is trying to give the same answer to that real question.
I’m already thinking about how I could have made this more clear in the essay, so thank you for pressing on it for clarity. I was trying to say this, but not as clearly, when I wrote in the first paragraph that claims of cosmic Schelling goodness are “claims about a class of hypothetical coordination games in the sense of Thomas Schelling”. The claims are made in reality by real agents (like me!), but the claims are about hypothetical scenarios where everyone is trying to give the same answer.
cosmic Schelling norms are just a subset of cosmic Schelling answers (“to a pro tanto moral question”)
Yep that is what I mean!
and therefore not necessarily actual norms in the dictionary sense of “a principle of right action binding upon the members of a group and serving to guide, control, or regulate proper and acceptable behavior”
I disagree with the “therefore not” here. Some norms have few adherents, and some have more. In computer science and math, the empty set is a set, and I think it makes sense to talk about a statement of type “norm” that might have effectively zero adherents, like “It’s good to ingest 3g of uranium mixed with applesauce on Tuesdays at 3:05pm”. That’s a norm that nobody follows, at least not on pre-2026 Earth before I wrote that sentence.
So I think cosmic Schelling norms are norms; they are statements of type “norm”.
But a cosmic Schelling norm is not necessarily a cosmically prevalent norm, in the sense of prevailing strongly over other pressures on behavior throughout the cosmos. Perhaps this is what you were pointing at when you said they are “not really norms”. Certainly, the central examples of “norms” that people think of are the prevalent ones, where the group of beings in some sense “bound” by the norm is substantial or relevant.
Still, even in a real human community, a norm can be broadly recognized as Schelling while not prevailing. Where I grew up, there were communities where “premarital sex is bad” was definitely the Schelling answer to “is premarital sex good or bad”: if you asked it on a survey and told people to pick the same answer as everyone else, they’d pick “bad” and be confident they were winning the Schelling survey game. Yet this was not a prevalent norm: most of the people were in fact having premarital sex. The norm did not prevail over other priorities, despite being Schelling and recognizably so amongst the group members.
That said, I’m pretty sure essentially every cosmic Schelling norm has some adherents, if only few.
your position as a compositional language realist
I’m quite pleased you saw this tweet and feel deeply understood by you mentioning it, as it absolutely applies here :)
Because metaethics considers spaces of norms, I’m using “norm” as a type, with some norms possibly having no adherents. So, “stealing is bad” is a norm, and “stealing is good” is also a norm, albeit with fewer adherents. Exactly one of those two norms is the cosmic Schelling norm regarding stealing, unless there is an exact 50⁄50 tie between the two, which seems extremely unlikely to me.
I think this usage fits with what it means to be “a” norm, and fits with other cases of conflict between norms, like “it’s good for women to vote” and “it’s bad for women to vote”, both of which are norms that have had non-zero support at various times and places in human history.
Given this usage, does it makes sense to you now how the following four sets of beings can be different?
those beings who say “stealing is bad” is a cosmic Schelling norm, i.e. the cosmic Schelling answer to “Is stealing good or bad?”;
those beings who know with high confidence that “stealing is bad” is a cosmic Schelling norm (i.e., know that it’s the most common answer to the cosmic Schelling question of “Is stealing good or bad?”);
those beings who endorse “stealing is bad” being the cosmic Schelling norm; and
those beings who adhere to that norm to some degree, i.e., make some non-negligible general effort to avoid stealing, as opposed to the opposite.
I think if you can see the difference between those four sets, and how I am using “norm” as a type that defines a space of competing possibilities including both “stealing is bad” and “stealing is good”, then it should help clear some things up.
I’m still not very sure what you meant by “actual norm in reality”, if you didn’t mean “actually prevalent norm throughout the cosmos”, so LMK if I missed the point there.
Is your point trying to use the same definitions and ontology as in the post, and responding to a particular logical argument made within it?
If yes, perhaps it would help if you would pick one of the specific arguments you disagree with — say, the 5-part argument argument for “stealing is bad” as a cosmic Schelling norm — and identify what is the first statement or inference in it that you think is wrong.
If no, can you say what is your definition of “stealing” and “stealing is good”? The post gives a particular cosmically general definition of “stealing”, and also “is cosmically Schelling-good”, and a particular argument that stealing is cosmically Schelling-bad.
The post does not define “is good” — it only notes an asymmetry of encouragement effects between “is good” and “is bad” — so I’m not sure if you are intending to use the same concepts as the post to respond to its arguments, or if you want to redirect or broaden our focus to some other sense of what you mean by “stealing” and “coding agents” and “all the time”.
Yes to the former, no the latter. I was more trying to “sanity check” your overall approach against my intuitions and trying to explain why I feel very skeptical about it. But I can try to trace back what part of the post I start to disagree with:
I’m not sure if there’s an even earlier step, but here it jumps out at me that you seem to have chosen “stealing” as an example because it’s a highly salient moral question for humans, but it may well not be for other civilizations, like my hypothetical one. You seem to be implicitly assuming that everyone will try to converge on the same questions (because “cosmic Schelling question asks what answer beings would converge on when trying to converge on the same answer”, which seemingly wouldn’t make sense to do if others are not actually trying to converge on the same questions), whereas my intuition is that norms and coordination mechanisms in general may be highly context-dependent so this seems like an unjustified assumption.
Thanks for answering my questions, I am more oriented now!
I don’t think that’s the case. On the contrary, here is the causal history of how I chose stealing:
A few years ago I was writing up some groundwork for a mathematical formalization of embedded agents, involving an information-theoretic boundary that distinguishes the agent from its environment… possibly a soft/fuzzy boundary, but a boundary nonetheless. While writing this, I noticed that wherever agents manage to persist over time, there are norms or ‘tendencies’ to respect their boundaries, almost tautologically. This lead me to write my Boundaries sequence.
I then looked for existing English words for what it means to (not) push or pull things through boundaries in various ways, and (not) stealing was the word for (not) taking stuff out of someone else’s boundary. Then, I would occasionally try to point out to people why this made stealing a relatively simple and thus convergently agreeable norm, but they didn’t seem sufficiently familiar with the dynamics of Schelling points for my point to be conveyed easily.
So I decided to write a post explaining how Schelling dynamics have a role to play in certain classes of meta-moral judgements, with an application at a cosmic scale, since at that scale people often have a lot of doubt and confusion about morality.
So, I don’t think I was looking to justify stealing, but rather, looking for word to refer to a fairly basic boundary-theoretic norm.
No, this is quite carefully not assumed. Please see the section entitled “This essay is not very skimmable”, which is written to emphasize up-front the distinction between
thought experiment stipulations, versus
assertions about what large classes of real agents would say about those thought experiments.
The assumption of intentional convergence lives in the thought experiment stipulations, not assumptions about real agents. That is, the essay does not assume, as a belief about real-world agents, that they share an actual intention to converge on an answer. Although, some agents do have that intention, and such intentions might be made more prevalent as a result of the Schelling participation effect described in the post, in which case those intentions are a consequence of reasoning rather than an assumption about the reasoner.
I had seen that warning, and was trying to keep track of the distinction, but apparently still failed. To check my understanding now:
cosmic Schelling answers are hypothetical answers in thought experiments where we assume that everyone is trying to converge to the same answers on the same questions
cosmic Schelling norms are just a subset of cosmic Schelling answers (“to a pro tanto moral question”), and therefore not necessarily actual norms in the dictionary sense of “a principle of right action binding upon the members of a group and serving to guide, control, or regulate proper and acceptable behavior”
In other words, the cosmic Schelling norm of an arbitrary pro tanto moral question probably exists in platonic space, but in most cases this would not be an actual norm in reality because (among other potential reasons) most beings in the cosmos would not actually be trying to converge on this particular question. Is this correct?
(If so, I’m confused how this usage of “norm” squares with your position as a compositional language realist, since compositionally it seems like a statement of the form “X is a cosmic Schelling norm” should imply that X is a norm?)
Not quite. As I intend it: cosmic Schelling answers are real answers to real questions about hypothetical scenarios in which everyone is trying to give the same answer to that real question.
I’m already thinking about how I could have made this more clear in the essay, so thank you for pressing on it for clarity. I was trying to say this, but not as clearly, when I wrote in the first paragraph that claims of cosmic Schelling goodness are “claims about a class of hypothetical coordination games in the sense of Thomas Schelling”. The claims are made in reality by real agents (like me!), but the claims are about hypothetical scenarios where everyone is trying to give the same answer.
Yep that is what I mean!
I disagree with the “therefore not” here. Some norms have few adherents, and some have more. In computer science and math, the empty set is a set, and I think it makes sense to talk about a statement of type “norm” that might have effectively zero adherents, like “It’s good to ingest 3g of uranium mixed with applesauce on Tuesdays at 3:05pm”. That’s a norm that nobody follows, at least not on pre-2026 Earth before I wrote that sentence.
So I think cosmic Schelling norms are norms; they are statements of type “norm”.
But a cosmic Schelling norm is not necessarily a cosmically prevalent norm, in the sense of prevailing strongly over other pressures on behavior throughout the cosmos. Perhaps this is what you were pointing at when you said they are “not really norms”. Certainly, the central examples of “norms” that people think of are the prevalent ones, where the group of beings in some sense “bound” by the norm is substantial or relevant.
Still, even in a real human community, a norm can be broadly recognized as Schelling while not prevailing. Where I grew up, there were communities where “premarital sex is bad” was definitely the Schelling answer to “is premarital sex good or bad”: if you asked it on a survey and told people to pick the same answer as everyone else, they’d pick “bad” and be confident they were winning the Schelling survey game. Yet this was not a prevalent norm: most of the people were in fact having premarital sex. The norm did not prevail over other priorities, despite being Schelling and recognizably so amongst the group members.
That said, I’m pretty sure essentially every cosmic Schelling norm has some adherents, if only few.
I’m quite pleased you saw this tweet and feel deeply understood by you mentioning it, as it absolutely applies here :)
Because metaethics considers spaces of norms, I’m using “norm” as a type, with some norms possibly having no adherents. So, “stealing is bad” is a norm, and “stealing is good” is also a norm, albeit with fewer adherents. Exactly one of those two norms is the cosmic Schelling norm regarding stealing, unless there is an exact 50⁄50 tie between the two, which seems extremely unlikely to me.
I think this usage fits with what it means to be “a” norm, and fits with other cases of conflict between norms, like “it’s good for women to vote” and “it’s bad for women to vote”, both of which are norms that have had non-zero support at various times and places in human history.
Given this usage, does it makes sense to you now how the following four sets of beings can be different?
those beings who say “stealing is bad” is a cosmic Schelling norm, i.e. the cosmic Schelling answer to “Is stealing good or bad?”;
those beings who know with high confidence that “stealing is bad” is a cosmic Schelling norm (i.e., know that it’s the most common answer to the cosmic Schelling question of “Is stealing good or bad?”);
those beings who endorse “stealing is bad” being the cosmic Schelling norm; and
those beings who adhere to that norm to some degree, i.e., make some non-negligible general effort to avoid stealing, as opposed to the opposite.
I think if you can see the difference between those four sets, and how I am using “norm” as a type that defines a space of competing possibilities including both “stealing is bad” and “stealing is good”, then it should help clear some things up.
I’m still not very sure what you meant by “actual norm in reality”, if you didn’t mean “actually prevalent norm throughout the cosmos”, so LMK if I missed the point there.