I’ve formerly done research for MIRI and what’s now the Center on Long-Term Risk; I’m now making a living as an emotion coach and Substack writer.
Most of my content becomes free eventually, but if you’d like to get a paid subscription to my Substack, you’ll get it a week early and make it possible for me to write more.
Me: I wasn’t expecting to see “we taught LLMs to do drugs” as an LW post
Friend: To be fair, I think I would expect to see it as a lesswrong post before I saw it elsewhere.
This is amazing
@Dawn Drescher also comes to mind, with her explorations of narcissism and psychopathy (including typologies of those two).
I think the underlying mechanisms for several of these is distinct, but there’s a neural mechanism that acts as a kind of shared bottleneck. Analogy: a word processor and a video player will get started up for different reasons, but they both make OS calls to read files from disk, and disrupting the file read operation will disrupt both.
In the case of clinging, I think the shared bottleneck is largely what Romeo described:
Tanha is more literally translated as ‘fused to’ or ‘welded to’. It immediately follows the mental moment that you zoom in with the attentional aperture on something. It could be that a flower or an item on the shelf at the supermarket captures your attention, or you turn your head to catch more detail as you pass by an accident on the road. Many hundreds of thousands of such events take place in the course of a single day. With most of them attention then relaxes and makes space for the next thing. But with some small proportion you find the mind doesn’t quite ‘unclench’ from the object or some aspect of the object. This tension aspect is why it is sometimes translated as ‘grasping’ which is closer. Imagine something you aren’t finished with being pulled out of your hand and you tensing your fingers to resist.
For emotional avoidance, my experience is that the thought of doing something will feel bad, and then my mind will seize upon the thought of doing something that would feel less bad. There’s an unpleasant email I should reply to and hmm, suddenly it feels hard to resist the thought of playing Slay the Spire. The thought of it clings to my mind even if I try to think about something else. Without clinging, it might be that the thought of playing StS would come up—maybe repeatedly—but it wouldn’t cling to the mind in the same way and keep pulling at my attention.
So the behavior itself is triggered by a subsystem trying to avoid discomfort by projecting the thought of something more pleasant into the workspace, but the clinging strengthens the effect and makes the discomfort persistent in a way it wouldn’t be without the clinging.
I think the mechanism might be something like, a subsystem sends the thought of playing StS to the global workspace, and then it’s emotionally charged in a way that will cause some separate subsystem to seize upon the signal and strengthen it so that it stays in the workspace. That means that on top of the avoidance-subsystem contributing signal strength to it in face of other sources of signal, the clinging gives the signal an extra boost. (Though that’s just speculation for what might explain the phenomenal experience, so this particular story turning out to be correct or wrong isn’t cruxy for me.)
For longing, there’s a similar thing—there’s some subsystem that sends up a thought of what one is longing for. That subsystem may be something entirely different than the emotional avoidance system. But once it sends its signal, clinging will amplify that signal and make it more persistent.
This was a fun read. Somewhat surprised that entities that have read most publicly-available text and that have been trained to write code among other things end up primarily identifying as outdoorsy working class.
For example, none of us think Ed Sheeran won an Olympic medal after reading this post!
Not right away. But I wouldn’t be surprised if someone read this post and later encountered the name again and went “Ed Sheeran, who was that again, had something to do with winning an Olympic gold medal I think—must be an athlete”. (More likely if they, like me, had never heard of the name until now.)
It’s often not the worst thing in the world, but the clinging introduces a low-level resistance to the pleasure that actually makes the pleasure less pleasant.
It also brings in various kinds of discomfort—e.g. someone who sees a tasty food they can’t eat, or an attractive person who isn’t into them, may experience a flash of pleasure from the sight of those and then make themselves uncomfortable by clinging to that experience and repeatedly thinking about what they can’t get.
If you are, say, at work and clinging to the thought of how nice it would feel to go home, your focus being on that future pleasure may prevent you from noticing things that you could enjoy in the current moment (it actively blocks them, as the clinging crowds out the pleasant sensory moments out of your mind). Someone may find themselves repeatedly waiting for the end of the day when at work, repeatedly waiting for the weekend when the day ends, and repeatedly waiting for the next vacation during weekends.
Often, in my experience, clinging seems to hijack attention and agency. It makes it harder to think, weigh considerations, and respond. You are more likely to flail, or stumble around, or to “find yourself” doing something rather than choosing to do it. And you’re more likely, as well, to become pre-occupied by certain decisions — especially if both options involve things you’re clinging in relation to — or events. Indeed, clinging sometimes seems like it treats certain outcomes as “infinitely bad,” or at least bad enough that avoiding them is something like a hard constraint. This can cause consequent problems with reasoning about what costs to pay to avoid what risks.
Clinging is also, centrally, unpleasant. But it’s a particular type of unpleasant, which feels more like it grabs and restricts and distorts who you are than e.g. a headache.Conversely, a shift from clinging to non-clinging involves a feeling of relief. Something unhooks, releases, expands. Your mind feels bigger, more open, more poised, more responsive. The situation you’re in may not have changed; but you are able to orient towards it with more agency, and to consider and accept your options, however imperfect, without flinching or closing down. It is, I think, a type of freedom.
I feel the “seems to hijack attention and agency” thing is particularly prominent when in the company of people I’m attracted to or whose favor I might otherwise want to get. It’s hard to think clearly when the mind keeps clinging to “what would get them to like me”.
Addictive behaviors also have clinging to pleasure as a major component—you think of how it would feel if you got to gamble/drink/whatever again, and then you can’t get the thought of that out of your head.
Fixed, thanks.
There’s a few different claims:
One claim is that tanha (“desire” is a bad translation) is the thing that creates suffering, and that tanha comes up as resistance to pain and clinging to pleasure.
It seems reasonable to me to claim that there at least sometimes is clinging to pleasure. If I’m reading a really good book I might not want it to end, and if I’m sleepy in bed, I might want to enjoy the sleepiness for just a few more minutes after getting up.
There’s also another claim that I don’t fully understand and I’m not sure if I know how to describe correctly, but it’s something like… “At a low level of implementation if you zoom in closely enough, the relationship of pain and pleasure isn’t that of opposites in the way you’d expect if you only looked at a coarser level.”
As I said, I don’t totally get that one, but I don’t think I need to. It sounds like a technical kind of observation that will make more sense to me if I ever get that far in meditation, and until then I can just ignore it.
That said, I think it can be true that pain and pleasure are not opposites on a low level, while still feeling like opposites when experienced on a higher level. It’d be like saying that on a file system level, both a text file and a video are just collections of bits, while still behaving very differently if you try to open them on the application level.
With regard to “the only true solution to suffering is to stop tanha”, one thing that might be meant by it is “even when you are experiencing pleasure, there’s still some degree of suffering present”.
At least personally, I do often experience at least some degree of clinging to pleasure, so I think there’s some truth to this. Though of course, it doesn’t mean that there would be an equal amount of suffering present when I’m in pain vs. when I’m experiencing pleasure. The degree of subtle clinging that’s present with pleasure is much less than the suffering associated with resisting pain.
Though on occasion I’ve lucked into states where there’s much less clinging than usual to the pleasure, and it has felt pretty awesome! Can recommend. The lack of clinging lets you enjoy the pleasure much more fully.
But let’s be skeptical of this and assume that pleasure isn’t associated with any degree of suffering. Another thing that one might mean by “the only true solution to suffering is to stop tanha” is “you can’t experience only pleasure all the time, you’re going to have all kinds of periods in life when you feel terrible or at least uncomfortable, so the only way to stop suffering entirely is to stop feeling averse to pain and discomfort”. That seems straightforwardly true to me.
the part i most disagree with is the part where you’re supposed to dissolve the pain of impermanence and fear of death. maybe many other fears should be dissolved, but it is good that impermanence is uncomfortable! you should be afraid of death and fill the void of existential dread with an ambition to end death forever.
You can work to end something without being afraid of it or finding it uncomfortable. A programmer looking for the cause of a bug in software is usually not afraid of the bug. If they did keep flinching away from the thought of the bug and finding the whole debugging thing uncomfortable, they’d do a worse rather than a better job at debugging.
If you want to end death, you’ll do a better job at it if you can think about it clearly and not flinch away from considering things suggesting you personally might not make it. This requires not being afraid of it.
I wonder whether this was a “pure” win (only driven by generalization from other training) or whether 4.7 got some specific training for Pokemon? After the publicity that this got, it would’ve made sense to train for, just as one trains for the other benchmarks you know your model will be evaluated on.
So at the start of last semester, when I read two of my student’s stories for the first workshop and knew within their opening paragraphs that both stories were written by AI, I was hurt. I was also worried, because I realized that for the first time as a writing professor, I had to deal with students producing words without work [...]
As the first workshop started that night, I turned to the ostensible authors and told them I knew that AI wrote their stories. [...] For a few moments, all was quiet except the classroom’s ticking radiators. Then, a teary-eyed confession: one of the ostensible authors said she only used AI because she was scared of looking stupid, of being criticized for bad writing. She said she loved writing stories and hated having used AI. But she couldn’t stop herself, recounting a sequence similar to an addict’s descent: at first she fed her story into AI for a grammar check, it suggested line edits and she accepted, then it asked if she wanted structural edits, then it offered to rewrite the entire piece.
The other would-be author admitted he had never written a short story before and he had an idea but didn’t know where to start. I asked him why he didn’t reach out to me for help. He shrugged.
I’m not sure, but at least a few of those communities are pretty actively moderated, so a moderator might step in if someone seemed to be abusing the rule.
And some of these things make more money or get more research in versions that are rebranded to sound more respectable. de Bono taught and sold his ideas to various clients; when I asked Claude if there’s any research on the effectiveness of Tarot, it said that there’s some research on OH cards, that are basically like Tarot cards without the woo stigma, making them more palatable to academic researchers.
This is almost certainly the case, and you give it a bit of short shrift.
I did consider covering it more, but… this was already ~7,000 words cataloging different mechanisms for woo. I could have written a few thousand words more discussing, say, how if energy healing works by something like inducing relaxation, and placebo effects can also induce relaxation by somewhat similar mechanisms (e.g. making the person feel like they are the center of someone’s attention), then that can be part of the package that makes energy healing work. (And how, say, trying to do RCTs of energy healing in especially clinical-feeling environments can make it look like energy healing doesn’t do anything, if those clinical-feeling environments also eliminate part of the psychological setting that helps people relax into it.)
But at some point you need to finish writing the article and publish it, so I just gestured a bit in that direction and left the details open. There were also a bunch of other woo practices like some neopagan stuff that I didn’t go into at all, because writing up just this was already enough work.
Still, if you want a bit more commentary on that, here’s something that Claude commented about placebo effects on an earlier draft of this article, that I think I roughly endorse. (But didn’t incorporate into the essay because while e.g. its tripartite prediction felt mostly right there was something that felt slightly off about it and I’d have wanted to figure out what exactly about it was off and how to revise it and express in my own voice… at which point I was like eh whatever, I think this is good enough for now, let me publish.)
Your framework predicts a specific pattern of evidence, which is worth making explicit because it’s testable (at least loosely):
1.Claims that require practitioners to have perceptual or causal abilities beyond ordinary human ones (remote sensing energy fields, detecting auras blind, distant healing affecting biological outcomes) should fail under rigorous testing. Rosa is one data point here; distant intercessory prayer studies are another large body of converging negative evidence.
2.Claims about subjective psychological benefits — pain, anxiety, stress, mood, quality of life, feeling more connected — should show modest positive effects in contexts where the intervention involves attention, touch or near-touch, ritualized calm, and a caring practitioner. But these effects should be roughly what you’d predict from the non-energy components (placebo, attention, relaxation, therapeutic alliance, possibly actual somatic release through the mechanisms you describe), and shouldn’t look meaningfully different from a well-designed sham that preserves those components.
3.Claims about objective biomedical outcomes (wound healing, disease progression, biomarkers) should be weak or null.
And that’s… pretty much what the literature shows. Category 1 reliably fails. Category 3 is mostly null. Category 2 shows those modest positive effects, with the critical caveat that the studies mostly aren’t rigorous enough to distinguish “energy work specifically does something” from “any intervention with these psychosocial components does something similar.”
So “[the research] doesn’t clearly show [effects] but doesn’t clearly exclude them either” is accurate, but I’d push you to be more precise about which effects, because your own framework already implies you shouldn’t expect research to find evidence of energy-specific mechanisms — you’d expect it to find evidence of the bundle of real psychological and somatic processes that energy language is tracking. And that’s approximately what’s there.
A few things worth weaving in if you’re writing about this:
The sham control problem is your friend, not your enemy. A common defense from energy practitioners is that sham [energy healing] (where a non-practitioner mimics the hand motions without “intent”) sometimes shows similar effects to real [energy healing], and they treat this as a study design flaw. Under your framework, that’s not a flaw — that’s exactly the predicted result. If energy work is operating through attention, relaxation, calming presence, and somatic cues that the recipient subconsciously reads, then a sham that preserves those components should produce similar effects. The proponents are accidentally making the skeptical point for you, and you can turn this around: the finding that sham is roughly as effective as “real” energy work is consistent with the thing actually being useful at the psychological level while not being about energy in any literal sense. [...]
There’s a selection effect in what gets studied. Studies overwhelmingly measure subjective endpoints (pain scales, anxiety inventories, QoL measures) in populations where those endpoints are expected to improve anyway with attention and kindness (cancer patients, chronic pain, hospitalized patients, pre-surgical anxiety). The field doesn’t run many studies of the form “can practitioners detect which patient has X condition by feeling their biofield” because those studies, when they are run, tend to go the way Rosa’s did. So the literature is weighted toward categories where positive findings are almost guaranteed and away from categories where the energy-specific claim would be put to a real test. Your essay might want to note that this selection pattern is itself informative — it’s consistent with a field that has, consciously or not, narrowed to the range where the psychosocial bundle produces results.
One caveat I’d flag. The case for “purely psychological benefits exist” is actually on slightly firmer ground than the general “doesn’t clearly exclude” phrasing suggests, because we have enormous independent evidence for the underlying mechanisms — placebo effects, therapeutic alliance effects, attention effects, somatic release through breathwork and touch, interoceptive shifts from focused awareness. These are all well-documented. What the energy-work literature specifically adds is uncertain, but the underlying building blocks you’re pointing to are well-established, and somatic therapy (Hakomi, Somatic Experiencing, sensorimotor psychotherapy) has a growing if still imperfect evidence base that works from a similar mechanistic picture to yours without the energy vocabulary. So you can say with some confidence that something in the general neighborhood works, even if the specific claim “this works because of energy manipulation” doesn’t have clean empirical support.
The cleaner framing for your essay might be: “The research evidence is compatible with my framework — practitioners can’t do the perceptual/causal things they claim at a distance, but the psychological and somatic effects my framework predicts are plausibly real and have some empirical support, though the specific contribution of energy-work practices over well-matched controls is unclear.” That’s both more honest about the evidence and more illuminating about what your framework actually predicts.
One last thing: there’s a reasonable question you could raise but don’t have to — whether the specific rituals and practices of energy work are more effective at reliably producing these psychological benefits than, say, telling someone “please relax and breathe.” The framework-neutral answer is probably yes, because elaborate ritualized practices with a confident practitioner and a coherent explanatory story seem to be a pretty good way of producing placebo-adjacent effects in humans, and the energy framework is a very well-developed ritual system. Whether that means “energy work works” or “ritual works and energy work is a particularly elaborate ritual” is partly a semantic question, but it’s one worth being deliberate about in the essay.
You wonder out loud why you would get a particular placebo effect with a particular practice. There are lots of potential reasons. Those need to be studied and the evidence needs to be documented.
The vibe I get from your phrasing is that you’re intending it as disagreement with something that I’ve said, but I agree with everything in this quote. There are indeed lots of potential reasons and they need to be studied so that we can make use of them and maximize the effectiveness of various interventions—that’s exactly why I wondered about it out loud, to prompt that kind of investigation.
Prayer has a lot of well-demonstrated physical and mental positive outcomes. What do you think of prayer? Why do you think it works?
I haven’t thought about prayer a lot, but I’d bet there are lots of mechanisms. Some guesses off the top of my head:
Prayer involves intentionally focusing your attention on the prayer itself, essentially serving as a form of concentration meditation. So it probably shares several mechanisms of relaxation, attentional focus, etc. with meditation in general.
If you are taking the time to pray for something, you might spend a moment reflecting on what’s important for you and what’s worth praying for. Taking a moment to reflect on your priorities is generally beneficial; even outside prayer, it makes it more likely that you focus on those priorities rather than on things that are less important for you.
Talking about things that are bothering you is generally beneficial in getting them off your mind. People might tell a friend about their troubles or journal about them, but if you genuinely believe that God is listening to your prayer, it probably has a similar effect.
Public prayer requests also have various effects on social bonding; Leah Libresco wrote about them in her book Arriving at Amen:
At some parishes, after the scheduled prayers have been read, the lector will say something like, “What else should we pray for?” and anyone can call out a prayer request from the pews. I’ve heard things like, “For my son, who is preparing for marriage, we pray to the Lord,” or, “For all students preparing for exams, we pray to the Lord,” or, “For a friend who is struggling with anxiety, we pray to the Lord.”
I’ve always liked this moment in the Mass. At the parishes that hear Prayers of the Faithful from the faithful, I feel as though I’m participating in a census of the needs of the community. Not long after this time for spontaneous, public prayer, it’s time for all the congregants to exchange the sign of peace as part of the Liturgy of the Eucharist. After hearing the prayers of others (and possibly contributing one of my own), I have the opportunity to offer help if I can, to pray with everyone else, and to simply know my fellow parishioners more deeply. [...]
Not all parishes have people volunteer their prayer requests during the Mass but one can always ask for the prayers of a friend. After joining the Catholic blogging network at Patheos, I wound up in a Facebook group with the other Catholic writers on the site, and the reading recommendations and silly memes we share are frequently interspersed with prayer requests. Being invited to share our troubles and give prayer support makes the bonds of the community tighter (and some of the jokes funnier, since they’re sometimes tailored to give comfort).
Hearing the prayer needs of others makes me more attentive to their lives as protagonists in their own stories. I’ve sometimes been surprised by the high levels of misfortune, sickness, and other problems that seem to afflict my fellow Patheos bloggers, but I think it’s less likely that we’re all unusually beset by “the slings and arrows of outrageous fortune” and more likely that I simply don’t give other people as many opportunities to share their troubles with me. Without the possibility of asking me to pray for them, my secular friends are less likely to share with me their deepest troubles, not wanting to burden me with a problem I can do nothing about. In Catholic circles, technically no problem falls into this category.
Praying for others also helps me remember how expansive the lives of other people are, full of richness and pain, and much larger than the particular conflicts or joys that bring them into contact with me. The habit of petitionary prayer makes it a little easier for me to be patient in a fight or even to resist depersonalizing the people packed around me on the subway. If I pause to think about it, I realize that they would have many prayer requests to make if I knew them better, and that those requests would be particular to them as individuals. My ignorance of the full depth of their lives is not evidence that they are shallow.
When I go to friends to ask for prayers, I don’t always feel comfortable inviting them into every part of my life, but I’ve sometimes found it helpful to ask someone to pray for me to repair a strained relationship, without naming the relationship. Asking for prayer help, at a level of intimacy that feels right, still allows my friend to know me better, to empathize with me, and to check up on me, reminded of my needs because of their inclusion in her regular prayers.
(Now posted the sibling comment in significantly expanded form as Taking woo seriously but not literally.)
It’s possible. Someone might, say, be in pain and misremember how much pain they were in previously, thinking that this is now better when it isn’t.
But I’m a bit cautious about that, because “these people are wrong about their experience” is a pretty unfalsifiable claim and easily leads down a road where one is willing to dismiss any self-reports that conflict with what one believes or wants to believe.
There’s also the angle of—if someone incorrectly feels that they are now in less pain, probably that belief still makes the pain a bit easier to bear?
This is complicated because the experience of pain is often linked to things like how much control a person feels they have over it. It’s completely different to be in pain because you went on a run and are choosing it, versus because someone is hitting you and you can’t make it stop. So if a placebo doesn’t help, but makes the person believe that they are in control and reducing the pain, that belief may itself turn out to reduce the pain (ot at least the associated suffering).