I think that at least some politeness falls more under the category of language. Like, I’m in Mexico, and it’s often helpful for me to switch to Spanish. I’m totally manipulating my signals there, but it seems… fine? Like I just don’t see the Goodhart pressure appearing there at all. Saying “Gracias, ¡hasta luego!” instead of “Thank you, have a good day!” seems perfectly fine.
But some politeness very much does introduce Goodhart drift. “How dare you say that?! That’s so rude!” This is a weird signal suppression system that introduces what some folks near Toronto coined as “untalkaboutability” (read as: “un-talk-about-ability”).
Likewise with pretending to be friendly. Lots of shop owners here will call out to me as I pass saying something like “Hey! Hey there my friend! Tell me, where are you from?” The context makes it pretty obvious that they’re being friendly to hook me into their shop. But the reason the hook works at all is because of the plausible deniability that that’s their purpose. “Oh, don’t be like that! I’m just being friendly!” This is weaponization of signals of friendliness, which is possible because of the Goodhart drift applied to those signals.
But yeah, I have a question around language here, and cultural standards. Like shaking hands in North America vs. bowing in Japan. This is actually a better edge case than is Spanish: It seems fine to recognize and act on the cultural difference…
…unless I switch because I’m trying to make others feel more comfortable. At that point I’m focusing on the signal in order to manipulate the other, which starts to introduce Goodhart drift. The fact that my intentions are good or that this is common doesn’t save the signal from Goodhart’s Demon.
Whereas if I can focus on grokking the cultural difference, and then set that entirely aside and do what I feel like doing… I think something like that naturally results in the politeness that matters.
It seems fine to recognize and act on the cultural difference…unless I switch because I’m trying to make others feel more comfortable.
Isn’t the main point of acting on cultural differences to make others feel more comfortable? Or to show that you’re interested in/you care about their culture?
Also, another weird case I just thought of: one of the biggest functions of clothing is signalling. Probably most people should lean towards wearing what they feel like more, but having this as a general policy might be quite costly, because people judge a lot based on clothing.
I wonder if the underlying disagreement here is: you’re saying not to do things which consciously seem like signalling. And you say things like:
I have a very clear intuition of it. I can feel it. I can notice cases where it happens and where it’s not happening, and I can often mentally transform one into the other. I know a bunch of the inner work needed to do it.
But I don’t believe this claim from you, because I think that a large proportion of signalling involves unconscious calculations or self-deception, and it takes a huge amount of work to make those explicit. So the category of “signalling” may, because of that, seem more pervasive and deeper-rooted to me than it does to you.
But I don’t believe this claim from you, because I think that a large proportion of signalling involves unconscious calculations or self-deception, and it takes a huge amount of work to make those explicit.
Valentine is someone who spent a huge amount of work on that. He was CFAR’s head of curriculum. Later, he spent a lot of time meditating. Valentine is not someone who speaks here from a place of not having put in the work.
Isn’t the main point of acting on cultural differences to make others feel more comfortable? Or to show that you’re interested in/you care about their culture?
As viewed from the outside, yes.
I think navigating this truthfully feels different from that analysis on the inside though.
If I think “I’m going to make these people feel comfortable by matching their cultural norms”, this can often create the opposite effect. I described the dynamics of this in the OP.
The reason those norms help put people at ease is because of what they imply (signal) about a certain quality of attention and compatibility you’re bringing. If you just are attentive then that’ll emerge naturally. No reason to think explicitly about the norms.
This is a little like noticing how all things about love and romance are ultimately about sex, but how thinking about it that way can actually jam their ability to function properly. This isn’t to deny the centrality of evolutionary forces. It’s noticing how thinking about those forces while inside them can create loops that bring in influences you may not want. Hence the “Just be yourself” advice.
Probably most people should lean towards wearing what they feel like more, but having this as a general policy might be quite costly, because people judge a lot based on clothing.
Yep. And if you focus your attention on other people’s judgments this way, you totally summon Goodhart’s Demon.
So which do you want? The risk of paying a social cost for a while, or the risk of floating along in Goodhart drift?
[…] I think that a large proportion of signalling involves unconscious calculations or self-deception, and it takes a huge amount of work to make those explicit. So the category of “signalling” may, because of that, seem more pervasive and deeper-rooted to me than it does to you.
That’s not what’s going on here.
I’m guessing you think I’m talking about actually in fact dropping all signaling.
That’s definitely not what I mean. That doesn’t make sense to me. It’d be on par with “Stop being affected by physics.”
When I say “Drop attempts to signal”, I’m describing the subjective experience of enacting this shift as I currently understand it.
I mean the thing where, when sitting across from someone on a first date, I can track the thoughts that are about “making a good impression” and either lean into them or sort of drop them. The first one structurally creates problems. The second is less likely to.
On the inside it feels like going in the direction of just not caring about what impressions I do or don’t give her. Which is to say, on the inside it feels like dropping all attempts to signal.
But of course my body language and word choice and dress and so on will signal all kinds of things to her. I haven’t actually dropped all signaling, or even subconscious attempts to signal.
It’s just that by pointing this optimization force away from those signals, I can encourage them to reflect reality instead of the (possibly false) image of myself a part of me wants her to see.
And by holding such a policy in myself, the signals I end up sending will always systematically (at least in the limit) align with the truth of this transparency. Signaling non-deception by not deceiving. Focus — even subconscious — on signals just can’t beat this strategy for fidelity of transmission best as I can tell.
Which is to say, the strategy of “Drop all attempts to signal” is a signaling strategy.
…at least in one analysis. Because thinking of it that way makes it harder to use, it helps to reframe it.
But my guess is that this resolves the difference in perspective here between you and me. Yes?
>>…unless I switch because I’m trying to make others feel more comfortable.
>Isn’t the main point of acting on cultural differences to make others feel more comfortable? Or to show that you’re interested in/you care about their culture?
As viewed from the outside, yes.
I think a better way of saying it is ”...unless I switch because I’m trying to avoid making people uncomfortable”.
There are all sorts of instrumental goals like “making people feel comfortable” which can be valid to focus on in the moment, provided that it’s context appropriate and appropriately delimited. For example, to hit your target with a rifle you might focus on bringing the cross hairs over the bullseye… unless your scope isn’t sighted in, or you’re far enough that there’s windage and drop to account for, etc. If you’re familiar enough with long range rifle shooting you’ll factor in drop and “kentucky windage” intuitively, yet at close range your mind is going to be only on bringing the cross hairs to the target, and that’s fine. Similarly, you can absolutely aim to “make people comfortable”, so long as you are aware of the limitations of this alignment and don’t get stuck when it doesn’t fit. So long as you’re happy to provoke temporary discomfort when it’s necessary AND so long as “make them comfortable” translates automatically with “be nonthreatening” AND “be nonthreatening” includes the self awareness that if the other person looks afraid, your self perception of “non-threatening” can’t be trusted and you have to actually look inwards to address potential threats until either you find a problem to fix or else seeing you do so causes them to feel comforted and stop giving you that error signal… then you’re fine.
The problem comes in when you try to “get away”, because almost everything that succeeds at “getting away” from a particular stimulus fails to get towards the actual goal. As a physical analogy, it’s hard to “push rope” because there’s nothing constraining it to that particular away and it just buckles towards any of the easier ones. You actually can push on similarly flexible throttle cables, but only to the extent that they’re tightly encased in stiff shrouds which restrict the directions of “away” that work. It’s a fundamentally unstable thing, and if you try to “get away from them being uncomfortable”, you have to be damn sure you’re restraining the buckling mode of “get away from them showing discomfort, by hiding it instead”, and any other potential buckling modes. That’s not what you want anyway, so better just to pull towards the actual goal, as best as you can identify it. You don’t actually know what this is, and trying to get towards an ill defined thing helps you notice when it’s not defined enough, and that further focusing on what you want is needed.
Dacyn’s comment and your distillation is relevant here: “would you rather self deceive or [have unacceptably bad thing happen]?”
To the extent that the consequence is actually unacceptably bad, and the hypothetical actually free of third options, then you gotta choose to not die. In everything else, it’s an open question of whether you can afford enough slack to accept risking the bad thing, and whether you can find a non-self-deceptive option that runs the risk down low enough. This gets especially bad when people lack the concept that “discomfort can be necessary and good”, because then you can’t even run the calculation and have to always err on the side of “not taking risks” and never getting the second marshmallow. If this is the case, then the more sensitive you are (in the “instrumentation” sense, where “sensitive” is good), then the more pathological this becomes.
For example, if you’re sensitive to disapproval from your hypothetical girlfriend’s parents, you try to “stay polite” and “not be rude or disrespectful” which totally sound like good things and you can easily convince yourself that they are unalloyed good… except that “treating someone like they can’t handle a little offense” is actually pretty disrespectful too, and being the kind of person who is afraid to say necessary things when they’re “slightly uncomfortable” isn’t how you take care of your girlfriend and isn’t how you gain her parents respect and approval. Crank this to 11, and what do you see happening?
In real life, there’s no such thing as “you either have to self deceive or you die”, but there are situations where figuring out how stay honest and not die is beyond your ability, or “not worth the effort”, or maybe just “not something you currently see how to do”. Having your cake and eating it too is always better though, and these skills do generalize a good deal, so its worth putting some work into holding yourself to “hard mode” and developing both the skills to make it economical and the mental fortitude to keep the option on the table.
I generally refer to this as being “security limited”. If you’re insecure and “need” approval of your girlfriend’s parents, then you can’t do things that risk not getting it, and you’re running away from disapproval (“death”). If you’re secure enough to take these risks, then you can remain faithful to your goals of being good for your girlfriend (even when it risks her parents’ disapproval, at least in the short term), and also your goals of being properly recognized as such because those two goals are not fundamentally and unchangeably misaligned. It applies far beyond what most people would recognize as “insecurity driven”, but it’s actually the same damn thing, and without a good understanding of the pattern you’re trying to match to (and a way of handling this information that doesn’t make awareness costly), it often flies beneath detection.
Yep, I’m pretty uncertain too.
I think that at least some politeness falls more under the category of language. Like, I’m in Mexico, and it’s often helpful for me to switch to Spanish. I’m totally manipulating my signals there, but it seems… fine? Like I just don’t see the Goodhart pressure appearing there at all. Saying “Gracias, ¡hasta luego!” instead of “Thank you, have a good day!” seems perfectly fine.
But some politeness very much does introduce Goodhart drift. “How dare you say that?! That’s so rude!” This is a weird signal suppression system that introduces what some folks near Toronto coined as “untalkaboutability” (read as: “un-talk-about-ability”).
Likewise with pretending to be friendly. Lots of shop owners here will call out to me as I pass saying something like “Hey! Hey there my friend! Tell me, where are you from?” The context makes it pretty obvious that they’re being friendly to hook me into their shop. But the reason the hook works at all is because of the plausible deniability that that’s their purpose. “Oh, don’t be like that! I’m just being friendly!” This is weaponization of signals of friendliness, which is possible because of the Goodhart drift applied to those signals.
But yeah, I have a question around language here, and cultural standards. Like shaking hands in North America vs. bowing in Japan. This is actually a better edge case than is Spanish: It seems fine to recognize and act on the cultural difference…
…unless I switch because I’m trying to make others feel more comfortable. At that point I’m focusing on the signal in order to manipulate the other, which starts to introduce Goodhart drift. The fact that my intentions are good or that this is common doesn’t save the signal from Goodhart’s Demon.
Whereas if I can focus on grokking the cultural difference, and then set that entirely aside and do what I feel like doing… I think something like that naturally results in the politeness that matters.
Isn’t the main point of acting on cultural differences to make others feel more comfortable? Or to show that you’re interested in/you care about their culture?
Also, another weird case I just thought of: one of the biggest functions of clothing is signalling. Probably most people should lean towards wearing what they feel like more, but having this as a general policy might be quite costly, because people judge a lot based on clothing.
I wonder if the underlying disagreement here is: you’re saying not to do things which consciously seem like signalling. And you say things like:
But I don’t believe this claim from you, because I think that a large proportion of signalling involves unconscious calculations or self-deception, and it takes a huge amount of work to make those explicit. So the category of “signalling” may, because of that, seem more pervasive and deeper-rooted to me than it does to you.
Valentine is someone who spent a huge amount of work on that. He was CFAR’s head of curriculum. Later, he spent a lot of time meditating. Valentine is not someone who speaks here from a place of not having put in the work.
As viewed from the outside, yes.
I think navigating this truthfully feels different from that analysis on the inside though.
If I think “I’m going to make these people feel comfortable by matching their cultural norms”, this can often create the opposite effect. I described the dynamics of this in the OP.
The reason those norms help put people at ease is because of what they imply (signal) about a certain quality of attention and compatibility you’re bringing. If you just are attentive then that’ll emerge naturally. No reason to think explicitly about the norms.
This is a little like noticing how all things about love and romance are ultimately about sex, but how thinking about it that way can actually jam their ability to function properly. This isn’t to deny the centrality of evolutionary forces. It’s noticing how thinking about those forces while inside them can create loops that bring in influences you may not want. Hence the “Just be yourself” advice.
Yep. And if you focus your attention on other people’s judgments this way, you totally summon Goodhart’s Demon.
So which do you want? The risk of paying a social cost for a while, or the risk of floating along in Goodhart drift?
That’s not what’s going on here.
I’m guessing you think I’m talking about actually in fact dropping all signaling.
That’s definitely not what I mean. That doesn’t make sense to me. It’d be on par with “Stop being affected by physics.”
When I say “Drop attempts to signal”, I’m describing the subjective experience of enacting this shift as I currently understand it.
I mean the thing where, when sitting across from someone on a first date, I can track the thoughts that are about “making a good impression” and either lean into them or sort of drop them. The first one structurally creates problems. The second is less likely to.
On the inside it feels like going in the direction of just not caring about what impressions I do or don’t give her. Which is to say, on the inside it feels like dropping all attempts to signal.
But of course my body language and word choice and dress and so on will signal all kinds of things to her. I haven’t actually dropped all signaling, or even subconscious attempts to signal.
It’s just that by pointing this optimization force away from those signals, I can encourage them to reflect reality instead of the (possibly false) image of myself a part of me wants her to see.
And by holding such a policy in myself, the signals I end up sending will always systematically (at least in the limit) align with the truth of this transparency. Signaling non-deception by not deceiving. Focus — even subconscious — on signals just can’t beat this strategy for fidelity of transmission best as I can tell.
Which is to say, the strategy of “Drop all attempts to signal” is a signaling strategy.
…at least in one analysis. Because thinking of it that way makes it harder to use, it helps to reframe it.
But my guess is that this resolves the difference in perspective here between you and me. Yes?
I think a better way of saying it is ”...unless I switch because I’m trying to avoid making people uncomfortable”.
There are all sorts of instrumental goals like “making people feel comfortable” which can be valid to focus on in the moment, provided that it’s context appropriate and appropriately delimited. For example, to hit your target with a rifle you might focus on bringing the cross hairs over the bullseye… unless your scope isn’t sighted in, or you’re far enough that there’s windage and drop to account for, etc. If you’re familiar enough with long range rifle shooting you’ll factor in drop and “kentucky windage” intuitively, yet at close range your mind is going to be only on bringing the cross hairs to the target, and that’s fine. Similarly, you can absolutely aim to “make people comfortable”, so long as you are aware of the limitations of this alignment and don’t get stuck when it doesn’t fit. So long as you’re happy to provoke temporary discomfort when it’s necessary AND so long as “make them comfortable” translates automatically with “be nonthreatening” AND “be nonthreatening” includes the self awareness that if the other person looks afraid, your self perception of “non-threatening” can’t be trusted and you have to actually look inwards to address potential threats until either you find a problem to fix or else seeing you do so causes them to feel comforted and stop giving you that error signal… then you’re fine.
The problem comes in when you try to “get away”, because almost everything that succeeds at “getting away” from a particular stimulus fails to get towards the actual goal. As a physical analogy, it’s hard to “push rope” because there’s nothing constraining it to that particular away and it just buckles towards any of the easier ones. You actually can push on similarly flexible throttle cables, but only to the extent that they’re tightly encased in stiff shrouds which restrict the directions of “away” that work. It’s a fundamentally unstable thing, and if you try to “get away from them being uncomfortable”, you have to be damn sure you’re restraining the buckling mode of “get away from them showing discomfort, by hiding it instead”, and any other potential buckling modes. That’s not what you want anyway, so better just to pull towards the actual goal, as best as you can identify it. You don’t actually know what this is, and trying to get towards an ill defined thing helps you notice when it’s not defined enough, and that further focusing on what you want is needed.
Dacyn’s comment and your distillation is relevant here: “would you rather self deceive or [have unacceptably bad thing happen]?”
To the extent that the consequence is actually unacceptably bad, and the hypothetical actually free of third options, then you gotta choose to not die. In everything else, it’s an open question of whether you can afford enough slack to accept risking the bad thing, and whether you can find a non-self-deceptive option that runs the risk down low enough. This gets especially bad when people lack the concept that “discomfort can be necessary and good”, because then you can’t even run the calculation and have to always err on the side of “not taking risks” and never getting the second marshmallow. If this is the case, then the more sensitive you are (in the “instrumentation” sense, where “sensitive” is good), then the more pathological this becomes.
For example, if you’re sensitive to disapproval from your hypothetical girlfriend’s parents, you try to “stay polite” and “not be rude or disrespectful” which totally sound like good things and you can easily convince yourself that they are unalloyed good… except that “treating someone like they can’t handle a little offense” is actually pretty disrespectful too, and being the kind of person who is afraid to say necessary things when they’re “slightly uncomfortable” isn’t how you take care of your girlfriend and isn’t how you gain her parents respect and approval. Crank this to 11, and what do you see happening?
In real life, there’s no such thing as “you either have to self deceive or you die”, but there are situations where figuring out how stay honest and not die is beyond your ability, or “not worth the effort”, or maybe just “not something you currently see how to do”. Having your cake and eating it too is always better though, and these skills do generalize a good deal, so its worth putting some work into holding yourself to “hard mode” and developing both the skills to make it economical and the mental fortitude to keep the option on the table.
I generally refer to this as being “security limited”. If you’re insecure and “need” approval of your girlfriend’s parents, then you can’t do things that risk not getting it, and you’re running away from disapproval (“death”). If you’re secure enough to take these risks, then you can remain faithful to your goals of being good for your girlfriend (even when it risks her parents’ disapproval, at least in the short term), and also your goals of being properly recognized as such because those two goals are not fundamentally and unchangeably misaligned. It applies far beyond what most people would recognize as “insecurity driven”, but it’s actually the same damn thing, and without a good understanding of the pattern you’re trying to match to (and a way of handling this information that doesn’t make awareness costly), it often flies beneath detection.