Fair. I was simply wondering whether or not you had something to back up your claim that this topic has been covered “quite extensively”.
The thing that backs it up is you looking literally at all. Anything that I suggest may not hit on the particular parts of the (underspecified) idea that are most salient to you and can therefore easily be dismissed out of hand. This results in a huge asymmetry of effort between me locating/recommending/defending something I think is relevant and you spending a single hour looking in the direction I pointed and exploring things that seem most relevant to you.
I would like to be clear that I do not intend to claim that Newcomblike suffering is fake in any way. Suffering is a subjective experience. It is equally real whether it comes from physical pain, emotional pain, or an initially false belief that quickly becomes true. Hopefully posting it in a place like Lesswrong will keep it mostly away from the eyes of those who will fail to see this point.
I am indifferent to the content of what you intend to claim! This is a difficult to topic to broach in a manner that doesn’t license people to do horrible things to themselves and others. The point I’m making isn’t that you are going to intentionally doing something bad; it is that I know this minefield well and would like to make you aware that it is, in fact, a minefield!
The LessWrong audience is not sanctified as the especially psychologically robust few. Ideas do bad things to people, and more acutely so here than in most places (e.g. Ziz, Roko). If you’re going to write a guide to a known minefield, maybe learn a thing or two about it before writing the guide.
I again ask though, how would a literature review help at all?
You are talking about something closely related to things a bunch of other people have talked about before you. Maybe one of them had something worthwhile to say, and maybe it’s especially important to investigate that when someone is putting their time into warning you that this topic is dangerous. Like, I obviously expected a fight when posting my initial comment, and I’m getting one, and I’m putting a huge amount of time into just saying over and over again “Please oh my god do not just pull something out of your ass on this topic and encourage others to read it, that could do a lot of damage, please even look in the direction of people who have previously approached this idea with some amount of seriousness.” And somehow you’re still just demanding that I justify this to you? I am here to warn you! Should I stand on my head? Should I do a little dance? Should I Venmo you $200?
Like, what lever could I possibly pull to get you to heed the idea that some ideas, especially ideas around topics like suffering and hyperstition, can have consequences for those exposed to them, and these can be subtle or difficult to point at, and you should genuinely just put any effort at all into investigating the topic rather than holding my feet to the fire to guess at which features are most salient to you and then orient an argument about the dangers in a manner that is to your liking?
I’m not sure how to feel about this general attitude towards posting. I think with most things I would rather err on the side of posting something bad; I think a lot of great stuff goes unwritten because people’s standards on themselves are too high.
Doesn’t apply when there are real dangers associated with a lazy treatment of a topic. Otherwise I just agree.
Beyond this, I think it’s the readers’ responsibility to avoid content that will harm them or others.
They will not know! It is your responsibility to frame the material in a way that surfaces its utility while minimizing its potential for harm. This is not a neutral topic that can be presented in a flat, neutral, natural, obvious way. It is charged, it is going to be charged, which sides are shown will be a choice of the author, and so far it looks like you’re content to lackadaisically blunder into that choice and blame others for tripping over landmines you set out of ignorance.
Again, I am a giant blinking red sign outside the suffering cave telling you ‘please read the brochure before entering the suffering cave to avoid doing harm to others,’ and you are making it my responsibility to convince you to read the brochure. From my perspective, you are a madman with hostages and a loaded gun! From your perspective, ignorant of the underspecified risks, I am wildly over-reacting. But you don’t know that you have a gun, and I am expensively penning a comment liable to receive multiple [Too Combative?] reacts because it is the most costly signal I know how to send along this channel. Please, dear god, actually look into it before publishing this post, and just try to see why these are ideas someone might think it’s worth being careful with!
Ok, I was probably not going to write the post anyway, but since no one seems to actively want it, your insistence that it requires this much extra care is enough to dissuade me.
I will say, though, that you may be committing a typical mind fallacy when you say “convincing is >>> costly to complying to the request” in your reply to Zack Davis’ comment. I personally dislike doing this kind of lit-review style research because in my experience it’s a lot of trudging through bullshit with little payoff, especially in fields like social psychology, and especially when the only guidance I get is “ask ChatGPT for related Buddhist texts”. I don’t like using ChatGPT (or LLMs in general; it’s a weakness of mine I admit). Maybe after a few years of capabilities advances that will change.
And it seems that I was committing a typical mind fallacy as well, since I implicitly thought that when you said “this topic has been covered extensively” you had specific writings in mind, and that all you needed to do was retrieve them and link them. I now realize that this assumption was incorrect, and I’m sorry for making it. It is clear now that I underestimated the cost that would be incurred by you in order to convince me to do said research before making a post.
I hope this concept gets discussed more in places like Lesswrong someday, because I think that there may be a lot of good we can do in preventing this kind of suffering, and the first step to solving a problem is pointing at it. But it seems like now is not the time and/or I am not the person to do that.
Thank you for this very kind comment! I would like to talk in more detail about what was going on for me here, because while your assumptions are kindly framed, they’re not quite accurate, and I think understanding a bit more about how I’m thinking about this might help.
The issue is not that I can’t easily think of things that look relevant/useful to me on this topic; the issue is that the language you’re using to describe the phenomenon is so different from the language used to describe it in the past that I would be staking the credibility of my caution entirely on whether you were equipped to recognizenearby ideas in an unfamiliar form — a form against which you already have some (justified!) bias. That’s why it would be so much work! I can’t know in advance if the Buddhist or Freudian or IFS or DBT or CBT or MHC framing of this kind of thing would immediately jump out to you as clearly relevant, or would help demonstrate the danger/power in the idea, much less equip you with the tools to talk about it in a manner that was sensitive enough by my lights.
So recommending asking ChatGPT wasn’t just lazily pointing at the lowest hanging fruit; the Conceptual-Rounding-Error-Generator would be extremely helpful in offering you a pretty quick survey of relevant materials by squinting at your language and offering a heap of nearby and not-so-nearby analogs. You could then pick the thing that you thought was most relevant or exciting, read a bit about it, and then look into cautions related to that idea (or infer them yourself), then generalize back to your own flavor of this type of thinking.
It’s simply not instructive or useful for me to try to cram your thought into my frame and then insist you think about it This Specific Way. Instead, noticing that all (or most) past-plausibly-related-thoughts (and, in particular, the thoughts that you consider nearest to your own) come with risks and disclaimers would naturally inspire you to take the next step and do the careful, sensitive thing in rendering the idea.
This is a hard dynamic to gesture at, and I did try to get it across earlier, but the specific questions I was being asked (and felt obligated to reply to) felt like attempts at taking short cuts that misunderstood the situation as something much simpler (e.g. ‘William could just tell me what to look at but he’s being lazy and not doing it’ or ‘William actually doesn’t have anything in mind and is just being mean for no reason’).
Hence my response of behaving unreasonably / embarrassing myself as a method of rendering a more costly signal. I did try to keep this from being outright discouraging, and hoped continuing to respond would generate some signal toward ‘I’m invested in this going well and not just bidding to shut you down outright.’
I think you should think more about this idea, and get more comfortable with shittier parts of connecting your ideas to broader conversations.
and you are making it my responsibility to convince you to read the brochure
I mean, yes? If you want someone to do something that they wouldn’t otherwise do, you need to persuade them. How could it be otherwise?
From my perspective, you are a madman with hostages and a loaded gun!
But this goes both ways, right? What counts as extortion depends on what the relevant property rights are. If readers have a right to not suffer, then authors who propose exploring suffering-causing ideas are threatening them; but if authors have a right to explore ideas, then readers who propose not exploring suffering-causing ideas are threatening them.
Interestingly, this dynamic is a central example of the very phenomenon Morphism is investigating! Someone who wants to censor an idea has a game-theoretic incentiveto self-modify to suffer in response to expressions of the the idea, in order to extort people who care about their suffering into not expressing the idea.
I am not experiencing suffering or claiming to experience suffering; I am illustrating that the labor requested of me is >>> expensive for me to perform than the labor I am requesting instead, and asking for some good faith. I find this a psychologically invasive and offensive suggestion on your part.
I mean, yes? If you want someone to do something that they wouldn’t otherwise do, you need to persuade them. How could it be otherwise?
In cases where convincing is >>> costly to complying to the request it’s good form to comply (indeed, defending this has already been more expensive for me than checking for pre-existing work would have been for the OP!).
I am not experiencing suffering or claiming to experience suffering [...] I find this a psychologically invasive and offensive suggestion on your part
Sorry, I should have been clearer: I was trying to point to the game-theoretic structure where, as you suggest by the “madman with hostages” metaphor, an author considering publishing an allegedly suffering-causing idea could be construed as engaging in extortion (threatening to cause suffering by publishing and demanding concessions in exchange for not publishing), but that at the same time, someone appealing to suffering as a rationale to not publish could be construed as engaging in extortion (threatening that suffering would be a result of publishing and demanding concessions, like extra research and careful wording, in exchange for publishing). I think this is an interesting game-theoretic consideration that’s relevant to the topic of discussion; it’s not necessarily about you.
In cases where convincing is >>> costly to complying to the request it’s good form to comply
How do we know you’re not bluffing? (Sorry, I know that’s a provocative-sounding question, but I think it’s actually a question that you need to answer in order to invoke costly signaling theory, as I explain below.)
Your costly signaling theory seems to be that by writing passionately, you can distinguish yourself as seeing a real danger that you can’t afford to demonstrate, rather than just trying to silence an idea you don’t like despite a lack of real danger.
When someone uses the phrase “costly signal”, I think it’s germane and not an isolated demand for rigor to point out that in the standard academic meaning of the term, it’s a requirement that honest actors have an easier time paying the cost than dishonest actors.
That is: I’m not saying you were bluffing; I’m saying that, logically, if you’re going to claim that costly signals make your claim trustworthy (which is how I interpreted your remarks about “a method of rendering a more costly signal”; my apologies if I misread that), you should have some sort of story for why a dishonest actor couldn’t send the same signal. I think this is a substantive technical point; the possibility of being stuck in a pooling equilibrium with other agents who could send the same signals as you for different reasons is definitely frustrating, but not talking about it doesn’t make the situation go away.
I agree that you’re free to ignore my comments. It’s a busy, busy world that may not last much longer; it makes sense that people to have better things to do with their lives than respond to every blog comment making a technical point about game theory. In general, I hope for my comments to provide elucidation to third parties reading the thread, not just the person I’m replying to, so when an author has a policy of ignoring me, that doesn’t necessarily make responding to their claims on a public forum a waste of my time.
In cases where convincing is >>> costly to complying to the request it’s good form to comply
This is about the most untrue and harmful thing I’ve seen written out in a while. Alice merely making a request does not obligate Bob to comply just because Bob complying is much easier than Alice convincing Bob to comply. Just no, you don’t wield that sort of power.
You’re generalizing to the point of absurdity, WAY outside the scope of the object-level point being discussed. Also ‘is good form’ is VERY far short of ‘obligated’.
Someone requested input on their idea and I recommended some reading because the idea is pretty stakes-y / hard to do well, and now you’re holding me liable for your maliciously broad read of a subthread and accusing me of attempting to ‘wield power over others’? Are you serious? What are the levers of my power here? What threat has been issued?
I’m going out on a limb to send a somewhat costly signal that this idea, especially, is worth taking seriously and treating with care, and you’re just providing further cost for my trouble.
The thing that backs it up is you looking literally at all. Anything that I suggest may not hit on the particular parts of the (underspecified) idea that are most salient to you and can therefore easily be dismissed out of hand. This results in a huge asymmetry of effort between me locating/recommending/defending something I think is relevant and you spending a single hour looking in the direction I pointed and exploring things that seem most relevant to you.
I am indifferent to the content of what you intend to claim! This is a difficult to topic to broach in a manner that doesn’t license people to do horrible things to themselves and others. The point I’m making isn’t that you are going to intentionally doing something bad; it is that I know this minefield well and would like to make you aware that it is, in fact, a minefield!
The LessWrong audience is not sanctified as the especially psychologically robust few. Ideas do bad things to people, and more acutely so here than in most places (e.g. Ziz, Roko). If you’re going to write a guide to a known minefield, maybe learn a thing or two about it before writing the guide.
You are talking about something closely related to things a bunch of other people have talked about before you. Maybe one of them had something worthwhile to say, and maybe it’s especially important to investigate that when someone is putting their time into warning you that this topic is dangerous. Like, I obviously expected a fight when posting my initial comment, and I’m getting one, and I’m putting a huge amount of time into just saying over and over again “Please oh my god do not just pull something out of your ass on this topic and encourage others to read it, that could do a lot of damage, please even look in the direction of people who have previously approached this idea with some amount of seriousness.” And somehow you’re still just demanding that I justify this to you? I am here to warn you! Should I stand on my head? Should I do a little dance? Should I Venmo you $200?
Like, what lever could I possibly pull to get you to heed the idea that some ideas, especially ideas around topics like suffering and hyperstition, can have consequences for those exposed to them, and these can be subtle or difficult to point at, and you should genuinely just put any effort at all into investigating the topic rather than holding my feet to the fire to guess at which features are most salient to you and then orient an argument about the dangers in a manner that is to your liking?
Doesn’t apply when there are real dangers associated with a lazy treatment of a topic. Otherwise I just agree.
They will not know! It is your responsibility to frame the material in a way that surfaces its utility while minimizing its potential for harm. This is not a neutral topic that can be presented in a flat, neutral, natural, obvious way. It is charged, it is going to be charged, which sides are shown will be a choice of the author, and so far it looks like you’re content to lackadaisically blunder into that choice and blame others for tripping over landmines you set out of ignorance.
Again, I am a giant blinking red sign outside the suffering cave telling you ‘please read the brochure before entering the suffering cave to avoid doing harm to others,’ and you are making it my responsibility to convince you to read the brochure. From my perspective, you are a madman with hostages and a loaded gun! From your perspective, ignorant of the underspecified risks, I am wildly over-reacting. But you don’t know that you have a gun, and I am expensively penning a comment liable to receive multiple [Too Combative?] reacts because it is the most costly signal I know how to send along this channel. Please, dear god, actually look into it before publishing this post, and just try to see why these are ideas someone might think it’s worth being careful with!
Ok, I was probably not going to write the post anyway, but since no one seems to actively want it, your insistence that it requires this much extra care is enough to dissuade me.
I will say, though, that you may be committing a typical mind fallacy when you say “convincing is >>> costly to complying to the request” in your reply to Zack Davis’ comment. I personally dislike doing this kind of lit-review style research because in my experience it’s a lot of trudging through bullshit with little payoff, especially in fields like social psychology, and especially when the only guidance I get is “ask ChatGPT for related Buddhist texts”. I don’t like using ChatGPT (or LLMs in general; it’s a weakness of mine I admit). Maybe after a few years of capabilities advances that will change.
And it seems that I was committing a typical mind fallacy as well, since I implicitly thought that when you said “this topic has been covered extensively” you had specific writings in mind, and that all you needed to do was retrieve them and link them. I now realize that this assumption was incorrect, and I’m sorry for making it. It is clear now that I underestimated the cost that would be incurred by you in order to convince me to do said research before making a post.
I hope this concept gets discussed more in places like Lesswrong someday, because I think that there may be a lot of good we can do in preventing this kind of suffering, and the first step to solving a problem is pointing at it. But it seems like now is not the time and/or I am not the person to do that.
Thank you for this very kind comment! I would like to talk in more detail about what was going on for me here, because while your assumptions are kindly framed, they’re not quite accurate, and I think understanding a bit more about how I’m thinking about this might help.
The issue is not that I can’t easily think of things that look relevant/useful to me on this topic; the issue is that the language you’re using to describe the phenomenon is so different from the language used to describe it in the past that I would be staking the credibility of my caution entirely on whether you were equipped to recognize nearby ideas in an unfamiliar form — a form against which you already have some (justified!) bias. That’s why it would be so much work! I can’t know in advance if the Buddhist or Freudian or IFS or DBT or CBT or MHC framing of this kind of thing would immediately jump out to you as clearly relevant, or would help demonstrate the danger/power in the idea, much less equip you with the tools to talk about it in a manner that was sensitive enough by my lights.
So recommending asking ChatGPT wasn’t just lazily pointing at the lowest hanging fruit; the Conceptual-Rounding-Error-Generator would be extremely helpful in offering you a pretty quick survey of relevant materials by squinting at your language and offering a heap of nearby and not-so-nearby analogs. You could then pick the thing that you thought was most relevant or exciting, read a bit about it, and then look into cautions related to that idea (or infer them yourself), then generalize back to your own flavor of this type of thinking.
It’s simply not instructive or useful for me to try to cram your thought into my frame and then insist you think about it This Specific Way. Instead, noticing that all (or most) past-plausibly-related-thoughts (and, in particular, the thoughts that you consider nearest to your own) come with risks and disclaimers would naturally inspire you to take the next step and do the careful, sensitive thing in rendering the idea.
This is a hard dynamic to gesture at, and I did try to get it across earlier, but the specific questions I was being asked (and felt obligated to reply to) felt like attempts at taking short cuts that misunderstood the situation as something much simpler (e.g. ‘William could just tell me what to look at but he’s being lazy and not doing it’ or ‘William actually doesn’t have anything in mind and is just being mean for no reason’).
Hence my response of behaving unreasonably / embarrassing myself as a method of rendering a more costly signal. I did try to keep this from being outright discouraging, and hoped continuing to respond would generate some signal toward ‘I’m invested in this going well and not just bidding to shut you down outright.’
I think you should think more about this idea, and get more comfortable with shittier parts of connecting your ideas to broader conversations.
I mean, yes? If you want someone to do something that they wouldn’t otherwise do, you need to persuade them. How could it be otherwise?
But this goes both ways, right? What counts as extortion depends on what the relevant property rights are. If readers have a right to not suffer, then authors who propose exploring suffering-causing ideas are threatening them; but if authors have a right to explore ideas, then readers who propose not exploring suffering-causing ideas are threatening them.
Interestingly, this dynamic is a central example of the very phenomenon Morphism is investigating! Someone who wants to censor an idea has a game-theoretic incentive to self-modify to suffer in response to expressions of the the idea, in order to extort people who care about their suffering into not expressing the idea.
I am not experiencing suffering or claiming to experience suffering; I am illustrating that the labor requested of me is >>> expensive for me to perform than the labor I am requesting instead, and asking for some good faith. I find this a psychologically invasive and offensive suggestion on your part.
In cases where convincing is >>> costly to complying to the request it’s good form to comply (indeed, defending this has already been more expensive for me than checking for pre-existing work would have been for the OP!).
Sorry, I should have been clearer: I was trying to point to the game-theoretic structure where, as you suggest by the “madman with hostages” metaphor, an author considering publishing an allegedly suffering-causing idea could be construed as engaging in extortion (threatening to cause suffering by publishing and demanding concessions in exchange for not publishing), but that at the same time, someone appealing to suffering as a rationale to not publish could be construed as engaging in extortion (threatening that suffering would be a result of publishing and demanding concessions, like extra research and careful wording, in exchange for publishing). I think this is an interesting game-theoretic consideration that’s relevant to the topic of discussion; it’s not necessarily about you.
How do we know you’re not bluffing? (Sorry, I know that’s a provocative-sounding question, but I think it’s actually a question that you need to answer in order to invoke costly signaling theory, as I explain below.)
Your costly signaling theory seems to be that by writing passionately, you can distinguish yourself as seeing a real danger that you can’t afford to demonstrate, rather than just trying to silence an idea you don’t like despite a lack of real danger.
But costly signaling only works when false messages are more expensive to send, and that doesn’t seem to be the case here. Someone who did want to silence an idea they didn’t like despite a lack of real danger could just as easily write as passionately as you.
I’m not trying to silence anything. I have really just requested ~1 hour of effort (and named it as that previously).
You’re hyperbolizing my gestures and making selective calls for rigor.
Meta: I hope to follow a policy of mostly ignoring you in the future, in this thread and elsewhere. I suggest allocating your energy elsewhere.
When someone uses the phrase “costly signal”, I think it’s germane and not an isolated demand for rigor to point out that in the standard academic meaning of the term, it’s a requirement that honest actors have an easier time paying the cost than dishonest actors.
That is: I’m not saying you were bluffing; I’m saying that, logically, if you’re going to claim that costly signals make your claim trustworthy (which is how I interpreted your remarks about “a method of rendering a more costly signal”; my apologies if I misread that), you should have some sort of story for why a dishonest actor couldn’t send the same signal. I think this is a substantive technical point; the possibility of being stuck in a pooling equilibrium with other agents who could send the same signals as you for different reasons is definitely frustrating, but not talking about it doesn’t make the situation go away.
I agree that you’re free to ignore my comments. It’s a busy, busy world that may not last much longer; it makes sense that people to have better things to do with their lives than respond to every blog comment making a technical point about game theory. In general, I hope for my comments to provide elucidation to third parties reading the thread, not just the person I’m replying to, so when an author has a policy of ignoring me, that doesn’t necessarily make responding to their claims on a public forum a waste of my time.
This is about the most untrue and harmful thing I’ve seen written out in a while. Alice merely making a request does not obligate Bob to comply just because Bob complying is much easier than Alice convincing Bob to comply. Just no, you don’t wield that sort of power.
You’re generalizing to the point of absurdity, WAY outside the scope of the object-level point being discussed. Also ‘is good form’ is VERY far short of ‘obligated’.
Someone requested input on their idea and I recommended some reading because the idea is pretty stakes-y / hard to do well, and now you’re holding me liable for your maliciously broad read of a subthread and accusing me of attempting to ‘wield power over others’? Are you serious? What are the levers of my power here? What threat has been issued?
I’m going out on a limb to send a somewhat costly signal that this idea, especially, is worth taking seriously and treating with care, and you’re just providing further cost for my trouble.