>Note that in this example, the “me” character does do narrative syncing – but in a way that does not disguise itself as information sharing about AI.
Well, it’s better, but in I think you’re still playing into [Alec taking things you say as orders], which I claim is a thing, so that in practice Alec will predictably systematically be less helpful and more harmful than if he weren’t [taking things you say as orders].
>How can a person shift things toward less confusing patterns?
[Tossing out ideas, caveat emptor]
You could draw attention in the moment to that dynamic, so that Alec can get introspection on the phenomenon, or metaphorically so that Alec can start to get a “brain-computer interface readout” on “right now I’m looking to be told what to do”.
You could abstractly clarify the situation (which this post helps with, thank you!).
You could do improv-y stuff, playing low status, playing like you’re looking for orders from Alec. (Warning, might be a kind of deception.)The supposed upside is to “shake things loose” a little so that Alec gets exposed to what it would be like if he was High and you were Low, so that when he’s back to his usual Low play, he can know it’s happening and know what’s different about it.
You could refuse to answer Alec until it seems like he’s acting like his own boss.
You could technically explain why you don’t want to give him orders.
You could address why Alec locally wants to be told what to do.
>Is it something we should simply try to minimize if we want good epistemics?
My wild guess is that first-order minimizing narrative-syncing isn’t that important, but being open to second-order corrections is very important, because that’s what determines your trajectory towards correct beliefs vs. delusion. There’s always a ton of narrative-syncing taken for granted, and primordially speech is corrections to that; just-trying-to-be-epistemic speech is a rarefied sort of thing. I don’t know how to balance being open to corrections against “losing momentum”, but the obvious guess is to not sacrifice any ability to correct at all, because subtle failures-to-correct can propagate as a narrative-sync to just not think about something. In other words, yay to simply minimizing pushes against correcting errors, boo to minimizing narrative-syncing.
>What’re some places where this happens a lot?
This happens a lot with a power / status differential. This isn’t a vacuous claim: one consequence is that I’m claiming that for each person Alec, there’s a roughly 1-dimensional value called “power” (well, maybe it depends on social context...) that Alec assigns to each other person Berry, and this “power” variable predicts all of: the extent to which Alec imputes narrative-syncingness onto Berry’s utterances; the extent to which Alec expects others to take Blob’s utterances as narrative-syncing; the extent to which Alec expects Berry to take others’s statements as narrative-syncing; the extent to which Alec, if so inclined, to punish deviations from those expectations.
>How can a person notice it?
If Berry “directs annoyance at” Alec in a way where the annoyance isn’t hidden but the reasons for the annoyance are hidden, that might be because Berry expected Alec to coordinate in a narrative-sync but Alec didn’t.
If Alec finds himself doing stuff on automatic, not really interested / curious / etc., that might be because he took orders from someone.
> “Why did I even sign up for this program?”
To say something maybe obvious, this seems like a great chance to pry your fingertips under the shield of narratives; why are you here Alec? You were expecting me to do something or know something? Here’s a description of some of the tacks I’ve taken in trying to solve this thing, and why they can’t work; what should we do?
>> You could refuse to answer Alec until it seems like he’s acting like his own boss.
Alternative suggestion: do not make your help conditional on Alec’s ability to phrase his questions exactly the right way or follow some secret rule he’s not aware of.
Just figure out what information is useful for newcomers, and share it. Explain what kinds of help and support are available and explain the limits of your own knowledge. The third answer gets this right.
The third answer gets it wrong if Alec takes it as an order as opposed to potentially useful information.
>Just figure out what information is useful for newcomers, and share it.
Yes, but this only makes sense if your statements are taken as information. If they aren’t, then the useful information is the fact that your statements aren’t being taken as information.
Well, it’s better, but in I think you’re still playing into [Alec taking things you say as orders], which I claim is a thing, so that in practice Alec will predictably systematically be less helpful and more harmful than if he weren’t [taking things you say as orders].
There seems to be an assumption here that Alec would do something relatively helpful instead if he weren’t taking the things you say as orders. I don’t think this is always the case: for people who aren’t used to thinking for themselves, the problem of directing your career to reduce AI risk is not a great testbed (high stakes, slow feedback), and without guidance they can just bounce off, get stuck with decision paralysis, or listen to people who don’t have qualms about giving advice.
Like, imagine Alec gives you API access to his brain, with a slider that controls how much of his daily effort he spends not following orders/doing what he thinks is best . You may observe that his slider is set lower than most productive people in AI safety, but (1) it might not help him or others to crank it up and (2) if it is helpful to crank it up, that seems like a useful order to give.
Anna’s Scenario 3 seems like a good way to self-consistently nudge the slider upwards over a longer period of time, as do most of your suggestions.
Good point. My guess is that if Alec is sufficiently like this, the right thing to do is to tell Alec not to work on AI risk for now. Instead, Alec, do other fun interesting things that matter to you; especially, try looking for things that you’re interested in / matter to you apart from social direction / feedback (and which aren’t as difficult as AI safety); and stay friends with me, if you like.
There definitely seem to be (relative) grunt work positions in AI safety, like this, this or this. Unless you think these are harmful, it seems like it would be better to direct the Alec-est Alecs of the world that way instead of risking them never contributing.
I understand not wanting to shoulder responsibility for their career personally, and I understand wanting an unbounded culture for those who thrive under those conditions, but I don’t see the harm in having a parallel structure for those who do want/need guidance.
That seems maybe right if Alec isn’t *interested* in helping in non-”grunt” ways. (TBC “grunt” stuff can be super important; it’s just that we seem much more bottlenecked on 1. non-grunt stuff, and 2. grunt stuff for stuff that’s too weird for people like this to decide to work on.) I’m also saying that Alec might end up being able and willing to help in non-grunt ways, but not by taking orders, and rather by going off and learning how to do non-grunt stuff in a context with more clear feedback.
It could be harmful to Alec to give him orders to work on “grunt” stuff, for example by playing in to his delusion that doing some task is crucially important for the world not ending, which is an inappropriate amount of pressure and stress and more importantly probably is false. It could potentially be harmful of Alec if he’s providing labor for whoever managed to gain control of the narrative via fraud, because then fraudsters get lots of labor and are empower to do more fraud. It could be harmful of Alec if he feels he has to add weight to the narrative that what he’s doing matters, thereby amplifying information cascades.
>Note that in this example, the “me” character does do narrative syncing – but in a way that does not disguise itself as information sharing about AI.
Well, it’s better, but in I think you’re still playing into [Alec taking things you say as orders], which I claim is a thing, so that in practice Alec will predictably systematically be less helpful and more harmful than if he weren’t [taking things you say as orders].
>How can a person shift things toward less confusing patterns?
[Tossing out ideas, caveat emptor]
You could draw attention in the moment to that dynamic, so that Alec can get introspection on the phenomenon, or metaphorically so that Alec can start to get a “brain-computer interface readout” on “right now I’m looking to be told what to do”.
You could abstractly clarify the situation (which this post helps with, thank you!).
You could do improv-y stuff, playing low status, playing like you’re looking for orders from Alec. (Warning, might be a kind of deception.)The supposed upside is to “shake things loose” a little so that Alec gets exposed to what it would be like if he was High and you were Low, so that when he’s back to his usual Low play, he can know it’s happening and know what’s different about it.
You could refuse to answer Alec until it seems like he’s acting like his own boss.
You could technically explain why you don’t want to give him orders.
You could address why Alec locally wants to be told what to do.
>Is it something we should simply try to minimize if we want good epistemics?
My wild guess is that first-order minimizing narrative-syncing isn’t that important, but being open to second-order corrections is very important, because that’s what determines your trajectory towards correct beliefs vs. delusion. There’s always a ton of narrative-syncing taken for granted, and primordially speech is corrections to that; just-trying-to-be-epistemic speech is a rarefied sort of thing. I don’t know how to balance being open to corrections against “losing momentum”, but the obvious guess is to not sacrifice any ability to correct at all, because subtle failures-to-correct can propagate as a narrative-sync to just not think about something. In other words, yay to simply minimizing pushes against correcting errors, boo to minimizing narrative-syncing.
>What’re some places where this happens a lot?
This happens a lot with a power / status differential. This isn’t a vacuous claim: one consequence is that I’m claiming that for each person Alec, there’s a roughly 1-dimensional value called “power” (well, maybe it depends on social context...) that Alec assigns to each other person Berry, and this “power” variable predicts all of: the extent to which Alec imputes narrative-syncingness onto Berry’s utterances; the extent to which Alec expects others to take Blob’s utterances as narrative-syncing; the extent to which Alec expects Berry to take others’s statements as narrative-syncing; the extent to which Alec, if so inclined, to punish deviations from those expectations.
>How can a person notice it?
If Berry “directs annoyance at” Alec in a way where the annoyance isn’t hidden but the reasons for the annoyance are hidden, that might be because Berry expected Alec to coordinate in a narrative-sync but Alec didn’t.
If Alec finds himself doing stuff on automatic, not really interested / curious / etc., that might be because he took orders from someone.
> “Why did I even sign up for this program?”
To say something maybe obvious, this seems like a great chance to pry your fingertips under the shield of narratives; why are you here Alec? You were expecting me to do something or know something? Here’s a description of some of the tacks I’ve taken in trying to solve this thing, and why they can’t work; what should we do?
>> You could refuse to answer Alec until it seems like he’s acting like his own boss.
Alternative suggestion: do not make your help conditional on Alec’s ability to phrase his questions exactly the right way or follow some secret rule he’s not aware of.
Just figure out what information is useful for newcomers, and share it. Explain what kinds of help and support are available and explain the limits of your own knowledge. The third answer gets this right.
> secret rule
It shouldn’t be secret.
The third answer gets it wrong if Alec takes it as an order as opposed to potentially useful information.
>Just figure out what information is useful for newcomers, and share it.
Yes, but this only makes sense if your statements are taken as information. If they aren’t, then the useful information is the fact that your statements aren’t being taken as information.
There seems to be an assumption here that Alec would do something relatively helpful instead if he weren’t taking the things you say as orders. I don’t think this is always the case: for people who aren’t used to thinking for themselves, the problem of directing your career to reduce AI risk is not a great testbed (high stakes, slow feedback), and without guidance they can just bounce off, get stuck with decision paralysis, or listen to people who don’t have qualms about giving advice.
Like, imagine Alec gives you API access to his brain, with a slider that controls how much of his daily effort he spends not following orders/doing what he thinks is best . You may observe that his slider is set lower than most productive people in AI safety, but (1) it might not help him or others to crank it up and (2) if it is helpful to crank it up, that seems like a useful order to give.
Anna’s Scenario 3 seems like a good way to self-consistently nudge the slider upwards over a longer period of time, as do most of your suggestions.
Good point. My guess is that if Alec is sufficiently like this, the right thing to do is to tell Alec not to work on AI risk for now. Instead, Alec, do other fun interesting things that matter to you; especially, try looking for things that you’re interested in / matter to you apart from social direction / feedback (and which aren’t as difficult as AI safety); and stay friends with me, if you like.
There definitely seem to be (relative) grunt work positions in AI safety, like this, this or this. Unless you think these are harmful, it seems like it would be better to direct the Alec-est Alecs of the world that way instead of risking them never contributing.
I understand not wanting to shoulder responsibility for their career personally, and I understand wanting an unbounded culture for those who thrive under those conditions, but I don’t see the harm in having a parallel structure for those who do want/need guidance.
That seems maybe right if Alec isn’t *interested* in helping in non-”grunt” ways. (TBC “grunt” stuff can be super important; it’s just that we seem much more bottlenecked on 1. non-grunt stuff, and 2. grunt stuff for stuff that’s too weird for people like this to decide to work on.) I’m also saying that Alec might end up being able and willing to help in non-grunt ways, but not by taking orders, and rather by going off and learning how to do non-grunt stuff in a context with more clear feedback.
It could be harmful to Alec to give him orders to work on “grunt” stuff, for example by playing in to his delusion that doing some task is crucially important for the world not ending, which is an inappropriate amount of pressure and stress and more importantly probably is false. It could potentially be harmful of Alec if he’s providing labor for whoever managed to gain control of the narrative via fraud, because then fraudsters get lots of labor and are empower to do more fraud. It could be harmful of Alec if he feels he has to add weight to the narrative that what he’s doing matters, thereby amplifying information cascades.