I also appreciate how it engages with Chesterton’s Fence and suggests e.g. “For instance, the closeted homophobe should probably move out of his homophobic social context if he can.”
I’m glad you appreciate it… but I’m honestly unsure how this is a Chesterton’s Fence thing. Can you explain? You mean e.g. I’m not proposing the closeted homophobe just force himself to become aware of his sexual orientation without getting out of dodge?
(FWIW: if that’s what you meant, then I’m accidentally applying Chesterton’s Fence! It’s more like, I think I see the reason behind the pattern, and if correct then he has to address the reason behind the pattern in order for the pattern to shift. The theory kind of states that he can’t knock down Chesterton’s Fence here.)
I wonder this post is an infohazard for people immersed in sufficiently strong social incentives. I know you acknowledge this, but still.. Have you tested these tools with people in very difficult socially contexts?
Not extensively. A little bit.
FWIW I kind of lean toward the camp of tongue-in-cheek thinking that the idea of infohazards is one of the few real examples we have of infohazards. I think a more helpful framing is that memes interact in complicated ways with memetic ecosystems (the way any organism interacts in complicated ways with ecosystems), and it’s possible to have an “invasive species” kind of effect from a meme, and some of those effects can be pretty bad for the ecosystem in question. And I view a human psyche as a kind of memetic ecosystem. So it’s more a matter of, what’s the memetic impact? How does the ecosystem adapt? Does it become more vibrant, or less? How shall we measure that vibrance?
I think that point adds relevant nuance.
My impression is that a policy of worrying about infohazards encourages a memetically unhealthy environment. It seems like care, but in practice what I find actually happens is that critical information gets blocked, the policy becomes a vector for status games (e.g. taking a patronizing view of others, like you know better than they do what’s good or bad for them), and the memetic adaptation that would organically happen in contact with the truth just… doesn’t happen. So you end up with lots of bizarre fictions embedded in the memetic context in question.
Like, I don’t know, it sure doesn’t seem to me like what originally happened around Roko’s Basilisk was all that sane. The original idea strikes me as, at most, a self-fulfilling prophecy. It encourages the thinker to embed themselves in the idea and immerse themselves in an existential horror. And sure, you can do that if you want. And sure, knowing that you could makes it more likely. But I dunno, is knowing about crystal meth a basilisk? I think Roko’s Basilisk is better thought of as a training opportunity: learn how to not negotiate with terrorist memes. How do you handle self-fulfilling prophecies without bricking your ability to prioritize truth?
There was a whole section of this essay I cut that’s exactly about this topic. I suspect that conscious hyperstitioning, and learning to switch which attractor one hyperstitions in a multistable space, is also a key rationality skill. You need it in order to make truth-seeking safe to do when seeing a bad possibility threatens to make that possibility more likely. Otherwise it becomes strategic to refuse to acknowledge certain possibilities. That skill helps navigate situations where you’re embedded and you either don’t notice or can’t disembed (which is the strategy the OP focuses on: kind of disembedding the conscious mind).
Anyway. I ramble a bit. My point is that I don’t think worrying about infohazards is a good general approach.
That said, there’s a question of whether introducing these ideas to someone in a great deal of social stress (e.g. someone in an intensely psychologically abusive relationship, like with a full-on narcissist who’s really good at reading people) would make their situation worse.
I haven’t tested that very much. It’s possible. It strikes me as quite unlikely. I think most such people just… won’t engage with the ideas.
But sure, strong psychotechnologies can have strong effects, some of which might be negative (possibly severely negative) on some people.
There was a very early version of this material that I ran on a group back in 2019. It didn’t have the “Chesterton’s Fence” aspects to it basically at all. It was more like a frontal assault on self-deception. It created some intense subjective experiences for many (most?) of the people in that fairly large audience. Most such effects were extremely positive over the following year or so. But one person had a reactivation of her suicidal tendencies that left her spinning for months; she eventually pulled out of it, but mostly by distracting herself from the possibility of using that early version of these tools.
(The early version was almost nothing like the ones I’ve presented in the OP by the way. I don’t think the OP’s tools are at all easy to misuse the way that the 2019 version was almost trivial to misuse.)
The thing is, even that one woman who became suicidal again had extra factors that this post just doesn’t have. She was embedded in a social context that pressured her for weeks after that lecture to use the tools. That’s when she became suicidal. And she stopped being suicidal when everyone backed off and she went back to distracting herself.
So I think the concern is more like, let’s not start a cult where we pressure each other to eliminate all of our protected problems. A supportive context that doesn’t feed social control disorders but nonetheless respects and welcomes them despite their protected problems seems maybe extremely good. But trying to challenge one another to stop rationalizing in specific ways strikes me as setting up terrible incentives.
But I don’t know. What would you like to see in terms of adequate care? I want things to be good for people, but I also don’t want fear to prevent us from developing potent psychotechnologies.
I’m glad you appreciate it… but I’m honestly unsure how this is a Chesterton’s Fence thing. Can you explain? You mean e.g. I’m not proposing the closeted homophobe just force himself to become aware of his sexual orientation without getting out of dodge?
(FWIW: if that’s what you meant, then I’m accidentally applying Chesterton’s Fence! It’s more like, I think I see the reason behind the pattern, and if correct then he has to address the reason behind the pattern in order for the pattern to shift. The theory kind of states that he can’t knock down Chesterton’s Fence here.)
Not extensively. A little bit.
FWIW I kind of lean toward the camp of tongue-in-cheek thinking that the idea of infohazards is one of the few real examples we have of infohazards. I think a more helpful framing is that memes interact in complicated ways with memetic ecosystems (the way any organism interacts in complicated ways with ecosystems), and it’s possible to have an “invasive species” kind of effect from a meme, and some of those effects can be pretty bad for the ecosystem in question. And I view a human psyche as a kind of memetic ecosystem. So it’s more a matter of, what’s the memetic impact? How does the ecosystem adapt? Does it become more vibrant, or less? How shall we measure that vibrance?
I think that point adds relevant nuance.
My impression is that a policy of worrying about infohazards encourages a memetically unhealthy environment. It seems like care, but in practice what I find actually happens is that critical information gets blocked, the policy becomes a vector for status games (e.g. taking a patronizing view of others, like you know better than they do what’s good or bad for them), and the memetic adaptation that would organically happen in contact with the truth just… doesn’t happen. So you end up with lots of bizarre fictions embedded in the memetic context in question.
Like, I don’t know, it sure doesn’t seem to me like what originally happened around Roko’s Basilisk was all that sane. The original idea strikes me as, at most, a self-fulfilling prophecy. It encourages the thinker to embed themselves in the idea and immerse themselves in an existential horror. And sure, you can do that if you want. And sure, knowing that you could makes it more likely. But I dunno, is knowing about crystal meth a basilisk? I think Roko’s Basilisk is better thought of as a training opportunity: learn how to not negotiate with terrorist memes. How do you handle self-fulfilling prophecies without bricking your ability to prioritize truth?
There was a whole section of this essay I cut that’s exactly about this topic. I suspect that conscious hyperstitioning, and learning to switch which attractor one hyperstitions in a multistable space, is also a key rationality skill. You need it in order to make truth-seeking safe to do when seeing a bad possibility threatens to make that possibility more likely. Otherwise it becomes strategic to refuse to acknowledge certain possibilities. That skill helps navigate situations where you’re embedded and you either don’t notice or can’t disembed (which is the strategy the OP focuses on: kind of disembedding the conscious mind).
Anyway. I ramble a bit. My point is that I don’t think worrying about infohazards is a good general approach.
That said, there’s a question of whether introducing these ideas to someone in a great deal of social stress (e.g. someone in an intensely psychologically abusive relationship, like with a full-on narcissist who’s really good at reading people) would make their situation worse.
I haven’t tested that very much. It’s possible. It strikes me as quite unlikely. I think most such people just… won’t engage with the ideas.
But sure, strong psychotechnologies can have strong effects, some of which might be negative (possibly severely negative) on some people.
There was a very early version of this material that I ran on a group back in 2019. It didn’t have the “Chesterton’s Fence” aspects to it basically at all. It was more like a frontal assault on self-deception. It created some intense subjective experiences for many (most?) of the people in that fairly large audience. Most such effects were extremely positive over the following year or so. But one person had a reactivation of her suicidal tendencies that left her spinning for months; she eventually pulled out of it, but mostly by distracting herself from the possibility of using that early version of these tools.
(The early version was almost nothing like the ones I’ve presented in the OP by the way. I don’t think the OP’s tools are at all easy to misuse the way that the 2019 version was almost trivial to misuse.)
The thing is, even that one woman who became suicidal again had extra factors that this post just doesn’t have. She was embedded in a social context that pressured her for weeks after that lecture to use the tools. That’s when she became suicidal. And she stopped being suicidal when everyone backed off and she went back to distracting herself.
So I think the concern is more like, let’s not start a cult where we pressure each other to eliminate all of our protected problems. A supportive context that doesn’t feed social control disorders but nonetheless respects and welcomes them despite their protected problems seems maybe extremely good. But trying to challenge one another to stop rationalizing in specific ways strikes me as setting up terrible incentives.
But I don’t know. What would you like to see in terms of adequate care? I want things to be good for people, but I also don’t want fear to prevent us from developing potent psychotechnologies.