It is widely known that cognitive behavioural therapy (CBT) is a standard treatment for depression. A little-known fact, however, is that its subcomponent, behavioural therapy (BT), is equally effective[1], according to a meta-analysis by Cochrane[2].
What the hell is that though?
If the term “reinforcement” makes you think of Sutton & Barto, or the newest shiny model release, then, ugh, fine, I’ll explain it in their terms:
Sequences of actions are graded according to a reward function. For example, DeepSeek spits out sequences of tokens, called chains-of-thought (CoT), and sequences that yield a correct answer are reinforced over those that do not.
Anything that is positively associated with getting the answer right, with getting the reward, such as correctly multiplying 322 by 214, is called a secondary reinforcer.
Behavioural therapy (BT) works by making effort a secondary reinforcer of your actions. That’s it.
To get started, you need morale[3], that is, you need to believe that effortful actions are associated with reward. If your level of morale is very low, you have come to believe there are no such actions, and taking very ambitious actions that then fail will only reinforce that belief[4].
Instead, morale needs to be raised in an incremental fashion. Behavioural therapy recommends the scheduling of activities here, meaning one sets out to do something they find fun, that also involves an amount of effort that is achievable. Once a simple activity has been successfully completed, one can plan an incrementally more effortful activity.
What gets humans reward?
If you are unsure what gives humans reward, please consult Maslow’s Hierarchy of Needs. Surely things like “having sex” or “gaining status” may not sound quite as idealistic as “making the singularity go well” to you.
But again, we can reach for the power of association: say there is a subculture with the aim of “making the singularity go well”, then “gaining status” may bring you closer to your terminal goal than initially assumed.
Once effort becomes habitual, you can leverage inner misalignment. That is, since secondary reinforcers are proxies for reward that become ingrained as habits over time, one is now able to perform the habit even though the original reward only shows up once in a while.
Since the rewards that humans are motivated by most strongly are unfortunately quite primitive and generally not helpful to the broader world, habits (as defined here) present an opportunity to act more idealistically in the long-term.
Environments and Effort Spirals
In fact, this principle extends way beyond any rigid DSM-5 definition[5], to all human behaviour. I posit the existence of two attractor states, existing alongside the middle ground of normalcy:
A laziness spiral[6], where one takes incrementally less effortful actions over time, which lowers morale recursively
An effort spiral[7], where one takes incrementally more effortful actions over time, which increases morale recursively
Any behaviour has a set of cues associated with it, e.g. a cue for DeepSeek is ‘322 × 214’. The environment is the set of all cues. Therefore, to change behaviour, a maximally disjoint environment needs to be created, whose cues must secondarily reinforce effort.
In simpler terms, the cues that trigger an unwanted behaviour must be removed, and replaced with an environment that incentivizes effort.
During the Manhattan Project, the Los Alamos Laboratory reliably created a strong effort spiral in its subjects, with ~75-hour work-weeks being common. Finding a modern equivalent is left as an exercise for the reader. Hint: Idk, do this “cold email” thing.
Finding a healthy, sustainable place on that spectrum, and avoiding the extremities[8] associated with both attractors is by no means an easy task. It can be reduced to finding the correct amount of slack in one’s life, and as much smarter ink has been spilled on this already[9], I invite you to find your own optimum.
Unfortunately, other factors also play a significant role. For instance, we know executive function[10] mediates the degree of effort one puts forth. The trait involves the ability to self-regulate and is highly heritable. The most effective intervention, stimulants, suffers from unclear long-term effectiveness[11].
On Punishment
Another slap in the face is that aversive stimulation (whose deliberate application by others is called dominance) may be useful[12].
A tiny bit of aversive stimulation by others is both ethical (don’t slap people!) and effective. A slight facial twitch signalling disapproval, or a downvote on a forum are good examples. This surgeon’s knife of behaviourism[13] is, however, available only for use by other people, and thus many alignment problems find their origin here. In the behaviouristic worldview, punishment by the self is oxymoronic, and I want to strongly discourage it.
All this leaves us with, well, my bad attempts at poetry:
At which point it is also prudent to clarify that this article does not aim to engage with the many subtleties of ‘depression’ as defined by the DSM-5.
Which John B. Watson, the father of behaviourism, had to of course use liberally by making a little child, Albert, afraid of a white rat, for science or something, you know? cf. Little Albert Experiment
Slack, And Slack Again, But Less And Less
It is widely known that cognitive behavioural therapy (CBT) is a standard treatment for depression. A little-known fact, however, is that its subcomponent, behavioural therapy (BT), is equally effective[1], according to a meta-analysis by Cochrane[2].
What the hell is that though?
If the term “reinforcement” makes you think of Sutton & Barto, or the newest shiny model release, then, ugh, fine, I’ll explain it in their terms:
Sequences of actions are graded according to a reward function. For example, DeepSeek spits out sequences of tokens, called chains-of-thought (CoT), and sequences that yield a correct answer are reinforced over those that do not.
Anything that is positively associated with getting the answer right, with getting the reward, such as correctly multiplying 322 by 214, is called a secondary reinforcer.
Behavioural therapy (BT) works by making effort a secondary reinforcer of your actions. That’s it.
To get started, you need morale[3], that is, you need to believe that effortful actions are associated with reward. If your level of morale is very low, you have come to believe there are no such actions, and taking very ambitious actions that then fail will only reinforce that belief[4].
Instead, morale needs to be raised in an incremental fashion. Behavioural therapy recommends the scheduling of activities here, meaning one sets out to do something they find fun, that also involves an amount of effort that is achievable. Once a simple activity has been successfully completed, one can plan an incrementally more effortful activity.
What gets humans reward?
If you are unsure what gives humans reward, please consult Maslow’s Hierarchy of Needs. Surely things like “having sex” or “gaining status” may not sound quite as idealistic as “making the singularity go well” to you.
But again, we can reach for the power of association: say there is a subculture with the aim of “making the singularity go well”, then “gaining status” may bring you closer to your terminal goal than initially assumed.
Once effort becomes habitual, you can leverage inner misalignment. That is, since secondary reinforcers are proxies for reward that become ingrained as habits over time, one is now able to perform the habit even though the original reward only shows up once in a while.
Since the rewards that humans are motivated by most strongly are unfortunately quite primitive and generally not helpful to the broader world, habits (as defined here) present an opportunity to act more idealistically in the long-term.
Environments and Effort Spirals
In fact, this principle extends way beyond any rigid DSM-5 definition[5], to all human behaviour. I posit the existence of two attractor states, existing alongside the middle ground of normalcy:
A laziness spiral[6], where one takes incrementally less effortful actions over time, which lowers morale recursively
An effort spiral[7], where one takes incrementally more effortful actions over time, which increases morale recursively
Any behaviour has a set of cues associated with it, e.g. a cue for DeepSeek is ‘322 × 214’. The environment is the set of all cues. Therefore, to change behaviour, a maximally disjoint environment needs to be created, whose cues must secondarily reinforce effort.
In simpler terms, the cues that trigger an unwanted behaviour must be removed, and replaced with an environment that incentivizes effort.
During the Manhattan Project, the Los Alamos Laboratory reliably created a strong effort spiral in its subjects, with ~75-hour work-weeks being common. Finding a modern equivalent is left as an exercise for the reader. Hint: Idk, do this “cold email” thing.
Finding a healthy, sustainable place on that spectrum, and avoiding the extremities[8] associated with both attractors is by no means an easy task. It can be reduced to finding the correct amount of slack in one’s life, and as much smarter ink has been spilled on this already[9], I invite you to find your own optimum.
Unfortunately, other factors also play a significant role. For instance, we know executive function[10] mediates the degree of effort one puts forth. The trait involves the ability to self-regulate and is highly heritable. The most effective intervention, stimulants, suffers from unclear long-term effectiveness[11].
On Punishment
Another slap in the face is that aversive stimulation (whose deliberate application by others is called dominance) may be useful[12].
A tiny bit of aversive stimulation by others is both ethical (don’t slap people!) and effective. A slight facial twitch signalling disapproval, or a downvote on a forum are good examples. This surgeon’s knife of behaviourism[13] is, however, available only for use by other people, and thus many alignment problems find their origin here. In the behaviouristic worldview, punishment by the self is oxymoronic, and I want to strongly discourage it.
All this leaves us with, well, my bad attempts at poetry:
The road from akrasia? — Well, it’s plain
and simple to express:
Slack
and slack
and slack again
but less
and less
and less[14].
The pharmacology people sometimes talk about “active ingredients” here, idk.
If the name “Cochrane” is unknown to you, just nod along approvingly, because they are very high-status wizards of meta-analytic black magic. cf. Behavioural therapies versus other psychological therapies for depression, Cochrane Database of Systematic Reviews
cf. Morale, J Bostock, LessWrong
A well-known psychiatrist says this is a common failure mode he experiences, so you better believe it, m’kay? cf. Peer Review Request: Depression, S. Alexander, AstralCodexTen, Point 2.1.5
At which point it is also prudent to clarify that this article does not aim to engage with the many subtleties of ‘depression’ as defined by the DSM-5.
cf. Laziness death spirals, PatrickDFarley, LessWrong
cf. Learned Industriousness, R. Eisenberger
For example, Oppenheimer dropped a substantial amount of weight due to working so much.
See for example On Slack, or Zvi’s longer considerations of this.
cf. Executive Functions: What They Are, How They Work, and Why They Evolved, R.A. Barkley
Which is the subject of, and a shameless ad for, my last article
cf. Dominance: The Standard Everyday Solution To Akrasia, johnswentworth, LessWrong
Which John B. Watson, the father of behaviourism, had to of course use liberally by making a little child, Albert, afraid of a white rat, for science or something, you know? cf. Little Albert Experiment
cf. LessWrong, 2012-present