One of the big differences between decoupling norms and contextualizing norms is that, in practice, it doesn’t seem possible to make statements with too many moving parts without contextualizers misinterpreting. Under contextualizing norms, saying “X would imply Y” will be interpreted as meaning both X and Y. Under decoupling norms, a statement like that usually means a complicated inference is being set up, and you are supposed to hold X, Y, and this relation in working memory for a moment while that inference is explained. There’s a communication-culture interpretation of this, where interpretations and expectations interact; if people translate “X would imply Y” into “X and Y”, and speakers anticipate this, then that’s what it means.
But I don’t think the communication culture interpretation correctly identifies what’s happening. I think what the phrase “contextualizing norms” points at is not a norm at all.
For most people, holding two sentences and a relationship between them in working memory, is not something they can do reliably. If you say “X would imply Y”, they will lose track of the relationship, and think that you said X directly and you said Y directly. Not because this is a correct interpretation under their communication culture, but because they literally can’t track the distinction, without slowing down or using a diagram or something.
This generalizes across most things that have subtlety. “Decoupling norms” make it easy to say things that contextualizing norms make it impossible to say.
When Eliezer wrote:
Make it explicit in international diplomacy that preventing AI extinction scenarios is considered a priority above preventing a full nuclear exchange, and that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.
What’s supposed to happen with this sentence, cognitively speaking, is that you read the sentence, slot it into a preexisting model of how deterrence and red lines work. If you don’t have that model, because you’re not an international relations expert and haven’t read Eliezer’s previous writing on the subject, then you have to hold onto the whole sentence, with all its moving parts and all its subtlety. This isn’t cognitively feasible for everyone, so they substitute a simpler sentence, and unfortunately in this case that means substituting something crazy that was not actually said.
The alternative would have been to embed a small lecture about international relations into the article. I think in this specific case that would have made things better, but that habitually dumbing things down in that way would be catastrophic. Because we still need to discover a solution to AI alignment, and I don’t think it’s possible to do that without discussing a lot of high-complexity things that can’t be said at all under contextualizing norms.
The alternative would have been to embed a small lecture about international relations into the article.
I don’t think that’s correct, there are cheap ways of making sentences like this one more effective as communication. (E.g. less passive/vague phrasing than “run some risk”, which could mean many different things.) And I further claim that most smart people, if they actually spent 5 minutes by the clock thinking of the places where there’s the most expected disvalue from being misinterpreted, would have identified that the sentences about nuclear exchanges are in fact likely to be the controversial ones, and that those sentences are easy to misinterpret (or prime others to misinterpret). Communication is hard in general, and we’re not seeing all the places where Eliezer did make sensible edits to avoid being misinterpreted, but I still think this example falls squarely into the “avoidable if actually trying” category.
habitually dumbing things down in that way would be catastrophic. Because we still need to discover a solution to AI alignment, and I don’t think it’s possible to do that without discussing a lot of high-complexity things that can’t be said at all under contextualizing norms
That’s why you do the solving in places that are higher-fidelity than twitter/mass podcasts/open letters/etc, and the communication or summarization in much simpler forms, rather than trying to shout sentences that are a very small edit distance from crazy claims in a noisy room of people with many different communication norms, and being surprised when they’re interpreted differently from how you intended. (Edit: the “shouting” metaphor is referring to twitter, not to the original Time article.)
What’s supposed to happen with this sentence, cognitively speaking, is that you read the sentence, slot it into a preexisting model of how deterrence and red lines work.
I think it’s a mistake to qualify this interpretation as an example of following decoupling norms. Deterrence and red lines aren’t mentioned in Eliezer’s comment at all; they’re just extra context that you’ve decided to fill in. That’s generally what people do when they read things under contextualizing norms. Interpreting this comment as a suggestion to consider initiating a nuclear exchange is also a contextualized reading, just with a different context filled in.
A highly-decoupled reading, by contrast, would simply interpret “some risk of nuclear exchange” as, well, some unquantified/unspecified risk.
Not because this is a correct interpretation under their communication culture, but because they literally can’t track the distinction, without slowing down or using a diagram or something.
I’m unsure but I suspect that in many cases you’re thinking of, this is incorrect. I think people can track implication when it’s something that “makes sense” to them, that they care about. I suspect that at least in a certain subset of these cases, what’s really happening is this:
They believe not-X (in some fashion). You start trying to say “X implies Y”. They get confused and are resistant to your statement. When pressed, it comes to light that they are refusing to think about possible worlds in which X is the case. They’re refusing because they believe not-X (in some fashion), and it’s pointless to think about worlds that are impossible—it won’t affect anything because it’s unreal, and it’s impossible to reason about because it’s contradictory. (They wouldn’t be able to say all of this explicitly.)
extremely insightful point, and for some reason it seems I deeply disagree with the aggregate point, but I can’t figure out why at the moment. strong upvoted though.
One of the big differences between decoupling norms and contextualizing norms is that, in practice, it doesn’t seem possible to make statements with too many moving parts without contextualizers misinterpreting. Under contextualizing norms, saying “X would imply Y” will be interpreted as meaning both X and Y. Under decoupling norms, a statement like that usually means a complicated inference is being set up, and you are supposed to hold X, Y, and this relation in working memory for a moment while that inference is explained. There’s a communication-culture interpretation of this, where interpretations and expectations interact; if people translate “X would imply Y” into “X and Y”, and speakers anticipate this, then that’s what it means.
But I don’t think the communication culture interpretation correctly identifies what’s happening. I think what the phrase “contextualizing norms” points at is not a norm at all.
For most people, holding two sentences and a relationship between them in working memory, is not something they can do reliably. If you say “X would imply Y”, they will lose track of the relationship, and think that you said X directly and you said Y directly. Not because this is a correct interpretation under their communication culture, but because they literally can’t track the distinction, without slowing down or using a diagram or something.
This generalizes across most things that have subtlety. “Decoupling norms” make it easy to say things that contextualizing norms make it impossible to say.
When Eliezer wrote:
What’s supposed to happen with this sentence, cognitively speaking, is that you read the sentence, slot it into a preexisting model of how deterrence and red lines work. If you don’t have that model, because you’re not an international relations expert and haven’t read Eliezer’s previous writing on the subject, then you have to hold onto the whole sentence, with all its moving parts and all its subtlety. This isn’t cognitively feasible for everyone, so they substitute a simpler sentence, and unfortunately in this case that means substituting something crazy that was not actually said.
The alternative would have been to embed a small lecture about international relations into the article. I think in this specific case that would have made things better, but that habitually dumbing things down in that way would be catastrophic. Because we still need to discover a solution to AI alignment, and I don’t think it’s possible to do that without discussing a lot of high-complexity things that can’t be said at all under contextualizing norms.
I don’t think that’s correct, there are cheap ways of making sentences like this one more effective as communication. (E.g. less passive/vague phrasing than “run some risk”, which could mean many different things.) And I further claim that most smart people, if they actually spent 5 minutes by the clock thinking of the places where there’s the most expected disvalue from being misinterpreted, would have identified that the sentences about nuclear exchanges are in fact likely to be the controversial ones, and that those sentences are easy to misinterpret (or prime others to misinterpret). Communication is hard in general, and we’re not seeing all the places where Eliezer did make sensible edits to avoid being misinterpreted, but I still think this example falls squarely into the “avoidable if actually trying” category.
That’s why you do the solving in places that are higher-fidelity than twitter/mass podcasts/open letters/etc, and the communication or summarization in much simpler forms, rather than trying to shout sentences that are a very small edit distance from crazy claims in a noisy room of people with many different communication norms, and being surprised when they’re interpreted differently from how you intended. (Edit: the “shouting” metaphor is referring to twitter, not to the original Time article.)
I think it’s a mistake to qualify this interpretation as an example of following decoupling norms. Deterrence and red lines aren’t mentioned in Eliezer’s comment at all; they’re just extra context that you’ve decided to fill in. That’s generally what people do when they read things under contextualizing norms. Interpreting this comment as a suggestion to consider initiating a nuclear exchange is also a contextualized reading, just with a different context filled in.
A highly-decoupled reading, by contrast, would simply interpret “some risk of nuclear exchange” as, well, some unquantified/unspecified risk.
I’m unsure but I suspect that in many cases you’re thinking of, this is incorrect. I think people can track implication when it’s something that “makes sense” to them, that they care about. I suspect that at least in a certain subset of these cases, what’s really happening is this:
They believe not-X (in some fashion). You start trying to say “X implies Y”. They get confused and are resistant to your statement. When pressed, it comes to light that they are refusing to think about possible worlds in which X is the case. They’re refusing because they believe not-X (in some fashion), and it’s pointless to think about worlds that are impossible—it won’t affect anything because it’s unreal, and it’s impossible to reason about because it’s contradictory. (They wouldn’t be able to say all of this explicitly.)
extremely insightful point, and for some reason it seems I deeply disagree with the aggregate point, but I can’t figure out why at the moment. strong upvoted though.