Antagonizing Opioid Receptors for (Prevention of) Fun and Profit
Related to: Ugh Fields, Are Wireheads Happy?
In his post Ugh Fields, Roko discussed “temporal difference learning”, the process by which the brain propagates positive or negative feedback to the closest cause it can find for the feedback. For example, if he forgets to pay his bills and gets in trouble, the trouble (negative feedback) propagates back to thoughts about bills. Next time he gets a bill, he might paradoxically have even more trouble paying it, because it’s become associated with trouble and negative emotions, and his brain tends to unconsciously flinch away from it.
He links to the associated Wikipedia article:
The TD algorithm has also received attention in the field of neuroscience. Researchers discovered that the firing rate of dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) appear to mimic the error function in the algorithm. The error function reports back the difference between the estimated reward at any given state or time step and the actual reward received. The larger the error function, the larger the difference between the expected and actual reward. When this is paired with a stimulus that accurately reflects a future reward, the error can be used to associate the stimulus with the future reward.
Dopamine cells appear to behave in a similar manner. In one experiment measurements of dopamine cells were made while training a monkey to associate a stimulus with the reward of juice. Initially the dopamine cells increased firing rates when exposed to the juice, indicating a difference in expected and actual rewards. Over time this increase in firing back propagated to the earliest reliable stimulus for the reward. Once the monkey was fully trained, there was no increase in firing rate upon presentation of the predicted reward. This mimics closely how the error function in TD is used for reinforcement learning.
So if I understand this right, the monkey hears a bell and is unimpressed, having no expectation of reward. Then the monkey gets some juice that tastes really good and activates (opioid dependent?) reward pathways. The dopamine system is pretty surprised, and broadcasts that surprise back to all the neurons that have been especially active recently, most notably the neurons that activated upon hearing the bell. These neurons are now more heavily associated with the dopamine system. So the next time the monkey hears a bell, it has a greater expectation of reward.
And in this case it doesn’t matter, because the monkey can’t do anything about it. But if it were a circus monkey, and its trainer was trying to teach it to do a backflip to get juice, the association between backflips and juice would be pretty useful. As long as the monkey wanted juice, merely entertaining the plan of doing a backflip would have motivational value that promotes the correct action.
The Sinclair Method is a promising technique for treating alcoholics that elegantly demonstrates these pathways by sabotaging them.
Alcohol produces a surge of opioids in, yes, the ventral tegmental area. The temporal difference algorithm there correctly deduces that the reward is due to alcohol, and so links the dopamine system to things like drinking, planning to drink, et cetera. Rounding the nearest cliche, dopamine represents “wanting”, so this makes people want to drink.
Repeat this process enough, or start with the right (wrong?) chemical structure for your opioid and dopamine receptors, and you become an alcoholic.
So to treat alcoholism, all you should have to do is reverse the process. Drink something, but have it not activate the reward system at all. Those dopaminergic neurons that detect error in your reward predictions start firing like mad and withdrawing their connections to the parts of the brain representing drinking, drinking is no longer associated with “wanting”, you don’t want to drink, and suddenly you’re not an alcoholic any more.
It’s not quite that easy. But it might be pretty close.
The Sinclair Method of treating alcoholism is to give patients naltrexone, an opioid antagonist. Then the patients are told they can drink as much as they want. Then they do. Then they gradually stop craving drink.
In these people, alcohol still produces opioids, but the naltrexone prevents them from working and they don’t register with the brain’s reward system. Drinking isn’t “fun” any more. The dopamine system notices there’s no reward, and downgrades the connection between reward and drinking, which from the inside feels like a lessened craving to drink.
In theory, this same process should be useful against any addiction or unwanted behavior. In practice, research either supports or is still investigating naltrexone use1 against smoking, self-harm, kleptomania, and overeating (no word yet on Reddit use).
The method boasts an success rate of between 25% to 78% on alcoholics depending on how you define success. A lot of alcoholism statistics are comparing apples to oranges (did they stay sober for more than a year? Forever? If they just lapsed once or twice, does that still count?) but eyeballing the data2 makes this look significantly better than either Alcoholics Anonymous or willpower alone.
I’m kind of confused by the whole idea because I don’t understand the lack of side effects. Knocking out the brain’s learning system to cure alcoholism seems disproportionate, and I would also expect naltrexone to interfere with the ability to experience happiness (which many people seem to like). But I haven’t heard anyone mention any side-effects along the lines of “oh, and people on this drug can never learn anything or have fun ever again”, and you’d think somebody would have noticed. If anyone on Less Wrong has ever used this method, or used naltrexone for anything else, please speak up.
Since these same pathways control so many cravings besides alcoholism, research in this area will probably uncover more knowledge of what really motivates us.
Footnotes
1: There’s a subtle but important difference between the Sinclair Method and simple naltrexone use. As I understand it, most doctors who prescribe naltrexone tell the patient to abstain from alcohol as much as possible, but the Sinclair Method tells the patients to continue drinking normally. There are also some complicated parts about exactly when and how often you take the drug. The theory predicts the Sinclair Method would have better results, and the data seems to bear this out. As far as I know, all the studies on kleptomania, overeating, et cetera have been done on standard naltrexone use, not the Sinclair Method; I predict the Sinclair Method would work better, although there might be some practical difficulties invovled in telling a kleptomaniac “Okay, take this tablet once a day while stealing stuff at the same rate you usually do.”
2: 27% “never relapse into heavy drinking” and 78% get drinking “below the level of increased risk of morbidity and mortality”. There’s also an 87% number floating around without any justification or link to a study. I think this guy’s statistics on a ~5-10% yearly remission rate from willpower or AA sound plausible.
Brain interventions are rife with cases where “somebody would have noticed” failed. For example, it was a long time before anybody noticed that people with frontal lobotomies had significant personality changes. And there is a famous surgeon, Ben Carson IIRC, who has done hemispherectomies, in which he removes one-half of a young child’s cortex. He claims this has no long-term harmful effect. I think he’s not looking hard enough.
(Also, the brain has numerous learning systems; opioid receptors are just one type. Even for a particular type of stimulus, learning and forgetting can use different systems, as they do for fear-learning. I would think you’d need to excite the forgetting system rather than knock out the learning system in this case.)
Fascinating,
Is there an opioid protagonist that makes some things “more rewarding”?
If you had a fast-acting antagonist/protagonist kit, you could train yourself to “love the jobs you hate”, Mr. Muscle style.
If I understand correctly, this suggestion is to combine opium (or other opiate) with a job of some kind. Probably any addictive drug (such as alcohol?) would work.
The trick isn’t combining the two—it’s making the two inseparable. I would expect that otherwise, people’s brains correctly solve the “credit assignment problem” and figure out which aspects of the combined experience are causing the pleasant feelings, and which are extraneous.
Possibly this is the analogue, in human-motivational-augmentation, of the wirehead problem in FAI.
The opposite of an antagonist is just something that attaches to the relevant receptors… so your suggestion seems to practically cash out as a very carefully managed use of things similar to heroine, morphine, or methadone? It might work, but I certainly wouldn’t want to be the first one to try it. It sounds like acquiring a habit in roughly the way a trained monkey does, performing the desired new behaviors for a jolt of positive reinforcement. Then make the reinforcement intermittent until you leave the trainee to their own devices. In this case, however, you’d be using opioids instead of fruit juice.
(One terminological quibble is that neurotransmitter “antagonists” block the action of “agonists”. It would make a sort of sense for “protagonist” to mean what “agonist” does within that technical area, but the relevant technical community’s jargon just doesn’t work that way.)
For a refinement, neurotransmitters floating free in one’s cerebro-spinal fluid take their “meaning” from the distribution of the cells that emit and detect a given kind of neurotransmitter and influence them via changing the ambient properties of the brain within the “channel” of that neurotransmitter… so perhaps a more careful implementation of the same idea would be to map the relevant parts of an individual’s brain very very carefully and come up with a wireheading system that wasn’t aimed at hedonism, but at habit or belief formation, by stimulating exactly the right parts of the brain at precisely the right moments to cause the appropriate kinds of general updates across the entire brain.
Perhaps it could even be done without “cutting open” anyone using neural magnetic stimulation but the engineering there looks pretty tricky to me. I’m not sure how precisely you could modulate the EM fields to focus on only the parts of the brain that one would be interested in. Again, I think I’d let others go first and see how it turns out.
A little piece of me fears the whole idea, because it sounds a little bit to me like dark side epistemology, only dressed in the clothes of science, medicine, and engineering, and promising a free ride to good character. It feels like a place where TANSTAAFL should apply somehow.
This comment isn’t specifically about naltrexone, and might not be relevant here.
This is an example that may provide some vague suggestive evidence that lack of opioids sometimes doesn’t help overcome addictions.
I used to really enjoy video games. I used to want to spend almost all my time playing video games. But then (for unrelated reasons?), I developed a bad case of anhedonia. Over a period of several years, I gradually lost the ability to experience pleasure, to the point where I was almost entirely incapable of experiencing any form of pleasure. Video games weren’t fun anymore. Playing them usually involved lots of stress, but no fun. But I was still addicted. The addiction got even worse when I started playing the Flash games at Kongregate. That site tracks your achievements across all the games, and often has challenges that are only available for a limited time. Now the previous addiction to positive experiences was replaced with an addiction to avoiding negative experiences. I was now addicted to not failing to earn these achievements, even I knew they really didn’t mean anything, and even though the games themselves involved lots of stress, and almost no fun at all. This continued for months, and the addiction didn’t end until after my anhedonia finally started to get less bad. Thanks Adelene! Thanks Alicorn!
So, yeah, that’s some anecdotal evidence that addictions can continue despite an almost complete absence of opioids.
What did Alicorn (or you and Alicorn) do to make the change?
Actually, Adelene Dawner was helping me for months before I had any contact with Alicorn, though Adelene’s methods were similar to what Alicorn described in her Luminosity Sequence.
Basically the main problem was that I was pushing myself too hard, and worrying too much, for reasons that were actually counterproductive. I was also trying to suppress all desires and emotion, and it turns out that was a very bad idea. I offered to post a detailed analysis of my progress, to LW, or possibly someplace else, but so far almost noone seemed interested.
As for the strategies we used to solve these problems, we got a surprising amount of leverage from the City of Lights technique. But mostly it was just asking the standard questions “what do I believe and why do I believe it?”, and “what am I doing and why am I doing it?”. And just generally talking through stuff we didn’t understand until we figured it out. And my results were surprisingly similar to what Alicorn described in her Ureshiku Naritai post.
Please upvote this comment if you downvoted the main comment, but for whatever reason want to balance out the effect on my karma score.
You say you decreased your anhedonia. Does that mean videogames are fun again?
I want to make my own anhedonia go away.
Interestingly, no, videogames are still usually more stressful than fun (with only a few specific exceptions, depending on how I feel at the moment). But other things are gradually becoming more enjoyable now. Most notably friendship. I sent you my contact info, I would like to chat with you about this.
From the Wikipedia article:
So it seems that the method is to time the disruption of opiod reinforcement to target when the person is drinking. Side effects may therefore be limited to other activities that typically coincide with drinking.
Though this also raises the issue of meta-akrasia, wouldn’t people learn to not like taking the tablets if they don’t enjoy anything they do for the next hour?
I don’t think so. This is the same idea as in my melatonin post about making yourself go to sleep; hyperbolic discounting means you can be rational about taking the pill well in advance, even if by the time it kicks in, your preferences have changed to prefer to have not taken the pill.
(And not enjoying things isn’t the same as displeasure or pain, after all. People don’t particularly enjoy most of what they do—it’s the extremes which are unusual & rare.)
Does it affect the feeling of enjoyment? I suspect it won’t, because enjoyment and reward are distinct.
I wonder if this stuff can be used as a diet pill?
It’s worth noting that for many alcoholics, “start of a drinking session” means “waking up in the morning”
Using that approach to quit smoking
This got me wondering… does procrastination involve opioids?
Was anyone able to find out the legal status in the US? It’s obviously permitted in some way, but in Googling, I couldn’t find its schedule.
The commercial name for naltrexone is Vivitrol, though that was a dead end for me for finding its legal status. It does note that it must be administered by a health professional.
Vivitrol is the extended release form of the drug. Since the Sinclair Method is sensitive to timing, I imagine that the usual Hydrochloride form is preferable, which goes by the brand names Revia or Depade. However, it’s already generic, so no need to pay for brand names!
It’s prescription only, but it’s not scheduled. The ease of obtaining it is likely dependent on how willing your doctor is to bend rules, or how well you can convince him or her that you are suffering from alcoholism. Even the experimental doctors want something plausible to write down however. They may be more persuaded by the mention of other applications of the drug, such as the trials on gambling addiction.
Because of the possible liver damage, as far as I can tell. I think one can (relatively) safely self-administer as long as one sticks to low doses; the principle says the effect ought to be weaker, not gone.
Did you ever look into this more?
No; way too speculative. If someone were to show any effectiveness outside two or three chemical addictions/substance abuse, then I might read up on it again.
See helpful comment by nickernst below.
I’ve talked to two people that have done this sort of thing for smoking, and it sorta worked.
They both said they lost interest in smoking and stopped, but the conversation came up because they were smoking and thinking about quitting again.
I’m curious why they started again, and if naltrexone administered before the first cigarette or two would have kept them from restarting the habit.
This method sounds like it could be useful for unconscious habits. I have a bad one of gnawing on my finger nails. By the time I realize I’m doing it, however, the damage has been done. For whatever reason, I think my brain has connected nail biting with stress release. Taking away that association without having to rely on my poor willpower would be nice.
In case you haven’t already tried it, denatonium would be much cheaper and less risky for your liver.
Magnesium & Calcium supplements are cheaper still—nail biting can be a symptom of a deficency in one or both. And one advantage of suppliments is that you should know pretty quickly if that’s the cause or not; in the admittedly few people I know who tried suppliments it either significantly reduced nail biting within a week or it never helped at all.
It can be really hard to tell whether a problem is more mind-like or more body-like.
Absolutely true. I test as I can, as difficult as that can be with a sample size of 1. But the magnesium/calcium supplement combo reliably stopped and restarted my nail-biting through three rounds of taking/not taking the supplements, which is as good a track record as I require. When I started testing my supplements I was not surprised to learn that several that had initially seemed beneficial didn’t provide any sustained benefit.
Could you help end news/TV enjoyment with this method? Take and then watch TV, play video games, browse reddit/CNN/etc. End political identity biases? Take and then go to a political party meeting, religious service or read a bunch of politically charged information. Food cravings? Take and stuff your face with ice cream and greasy mexican food. Dirty gossip? Take and gossip about who your friends have slept with, who hates who, etc.
Do any of these activites involve the endorphine mechanism related to drug use that it’s used for currently? Are you suggesting that this is a possible ticket to becoming a LessWrong superhero?
I don’t know. There’s no certain reason it shouldn’t work, but I could see a couple of possible sticking points.
Alcoholism’s a much more simple chemical addiction than some of the others. I don’t know if those are mediated by opioids, dopamine, or something else entirely (though it would be worth study). And if it works, without some countermethod like the one Roko’s hinting at in his comment, you’re going to kill your ability to enjoy everything you currently enjoy without gaining any new enjoyment of anything else, which sounds as likely to leave you depressed as superproductive.
Oh, and the alcoholics have to take the drug before each time they drink alcohol, even if it’s twenty years after the finish the original course and they’ve been sober the whole time. Otherwise they get the reinforcement and the alcoholism starts to return. So I doubt anyone could practically arrange to take a tablet a half hour before each time they think about in-group politics.
In a few years I might be in a better position to actually do some experiments with this stuff, at which point I’ll report back.
A different angle on alcoholism
The End of My Addiction is by a cardiologist who became an alcoholic and eventually found that Baclofen, an muscle relaxant, eliminated his craving for alcohol and surprisingly, his compulsive shopping.
His theory is that alcoholism is an effort to feel normal—taking away the pleasure from the alcohol isn’t a good solution.
I’ve heard good things about baclofen too, but I haven’t looked at in in depth. If you’ve read that book, I’d be really interested in seeing a post about the science behind it.
I didn’t remember enough to answer your question, but I found a summary, with self-experimentation anecdotes in the comments.
One matter of minutae here: the link to the “78% get drinking below the level of increased risk of morbidity and mortality.” I’d really love to see the original if you still have the link, Yvain. Thanks.
This post is timely. I’m an alcoholic who just began the Sinclair Method today. Subjectively, I don’t notice any anhedonia or lack of happiness. On the other hand, I’m still getting a buzz from the alcohol so maybe something’s gone wrong.
I actually asked the MD on staff in my rehab exactly the same question—“if it blocks dopamine, how am I ever supposed to feel happy ever again?” And he reassured me that it only blocked the reward from alcohol (and heroin) specifically. This feels too good to be true, but on the other hand the Sinclair Method works and anhedonia or depression aren’t commonly known side effects of Naltrexone, so it looks empirically plausible.