I think this theory matches my internal experience when I’m struggling to exert willpower. My intellectual/logical brain processes have some evidence for doing something (“knowing how the education system works, it’s important to do homework so I can get into a good college and get the job I want”). My reinforcement-learner/instinctual brain processes have some opposing argument (“doing your homework has never felt reinforcing in the past, but playing computer games has felt really reinforcing!”). These two processes fight it out. If one of them gets stronger (for example, my teacher says I have to do the homework tomorrow or fail the class) it will have more “evidence” for its view and win out.
It also explains an otherwise odd feature of willpower: sufficient evidence doesn’t necessarily make you do something, but overwhelming evidence sometimes does. For example, many alcoholics know that they need to quit alcohol, but find they can’t. They only succeed after they “hit bottom”, ie things go so bad that the evidence against using alcohol gets “beyond a reasonable doubt”. Alcoholism involves some imbalance in brain regions such that the reinforcing effect of alcohol is abnormally strong. The reinforcement system is always more convinced in favor of alcohol than the intellectual system is convinced against it—until the intellectual evidence becomes disproportionately strong even more than the degree to which the reinforcement system is disproportionately strong.
Me, later in my post:
Note those last sentences: besides the subsystems making their own predictions, there might also be a meta-learning system keeping track of which other subsystems tend to make the most accurate predictions in each situation, giving extra weight to the bids of the subsystem which has tended to perform the best in that situation. We’ll come back to that in future posts.
This seems compatible with my experience in that, I feel like it’s possible for me to change even entrenched habits relatively quickly—assuming that the new habit really is unambiguously better. In that case, while I might forget and lapse to the old habit a few times, there’s still a rapid feedback loop which quickly indicates that the goal-directed system is simply right about the new habit being better.
Or, the behavior in question might be sufficiently complex and I might be sufficiently inexperienced at it, that the goal-directed (default planning) subagent has always mostly remained in control of it. In that case change is again easy, since there is no strong habitual pattern to override.
In contrast, in cases where it’s hard to establish a new behavior, there tends to be some kind of genuine uncertainty:
The benefits of the old behavior have been validated in the form of direct experience (e.g. unhealthy food that tastes good, has in fact tasted good each time), whereas the benefits of the new behavior come from a less trusted information source which is harder to validate (e.g. I’ve read scientific studies about the long-term health risks of this food).
Immediate vs. long-term rewards: the more remote the rewards, the larger the risk that they will for some reason never materialize.
High vs. low variance: sometimes when I’m bored, looking at my phone produces genuinely better results than letting my thoughts wander. E.g. I might see an interesting article or discussion, which gives me novel ideas or insights that I would not otherwise have had. Basically looking at my phone usually produces worse results than not looking at it—but sometimes it also produces much better ones than the alternative.
Situational variables affecting the value of the behaviors: looking at my phone can be a way to escape uncomfortable thoughts or sensations, for which purpose it’s often excellent. This then also tends to reinforce the behavior of looking at the phone when I’m in the same situation otherwise, but without uncomfortable sensations that I’d like to escape.
When there is significant uncertainty, the brain seems to fall back to those responses which have worked the best in the past—which seems like a reasonable approach, given that intelligence involves hitting tiny targets in a huge search space, so most novel responses are likely to be wrong.
Also, you:
Me, later in my post: