I find the “Sweet delusions” paper to be quite unconvincing on a close reading. They use a very different task than any previous trial (selected for having a much, much high test-retest reliability—strongly suggesting that performance is less contingent on state!), but still suppose that the effect size of glucose depletion should be the same as in the literature (I have no idea why you would think this).
They find a large and statistically signifiant effect post-treatment—the glucose group has much higher willpower. But they also find a large and statistically significant effect pre-treatment, and so recourse to a more sophisticated analysis. This is inconsistent with the treatment effect observed in the original studies, but to be frank this is unsurprising given the high test-retest reliability of the measure they chose.
In fact it looks like they find substantially higher willpower in the second trial (with a larger improvement in the control group, going along with their much lower levels of initial willpower). This contradicts many well-replicated results, and seems like a good pointer that they may not be measuring willpower. It’s also quite easy to see a number of ways in which this could happen as an artifact of their methodology.
They later produced a second replication with the same subjects. They fail to find an effect, but they use “tendency to clear more lines at a time in tetris” as their measure of self-control, which as far as I know has never been used in another study and doesn’t seem very compelling to me. I don’t know why in god’s name they wouldn’t just replicate with any one of a dozen standard tests of willpower depletion. I would be curious if anyone has insight into this.
It looks like neither paper has any other citations, though they have seen a good bit of popular discussion. My best guess is that this is another example of it being easy for a crappy replication to fail to reproduce an effect regardless of whether it is real.
I remain agnostic about the original effect (at least relative to my disdain for this study). It looks to me like the evidence is reasonably good, and if true it certainly seems like an important fact. I’d be curious if anyone has thoughts on it.
ETA my best guess: willpower depletion is real, the perception of glucose (but not an artificial sweetener) really does help a lot with performance 10m later if in a depleted state, and recently agreeing to “I don’t think willpower depletion is real” causes people to put in much more effort when in a depleted state. The theory “people who believe in willpower depletion experience it more” does explain the latter observation, but the correlational studies seem to suffer badly from reverse causality, and the experimental studies seem to involve implausibly large effects given the posited mechanism, so I suspect that something else is going on (and a number of contenders do leap to mind).
I find the “Sweet delusions” paper to be quite unconvincing on a close reading. They use a very different task than any previous trial (selected for having a much, much high test-retest reliability—strongly suggesting that performance is less contingent on state!), but still suppose that the effect size of glucose depletion should be the same as in the literature (I have no idea why you would think this).
They find a large and statistically signifiant effect post-treatment—the glucose group has much higher willpower. But they also find a large and statistically significant effect pre-treatment, and so recourse to a more sophisticated analysis. This is inconsistent with the treatment effect observed in the original studies, but to be frank this is unsurprising given the high test-retest reliability of the measure they chose.
In fact it looks like they find substantially higher willpower in the second trial (with a larger improvement in the control group, going along with their much lower levels of initial willpower). This contradicts many well-replicated results, and seems like a good pointer that they may not be measuring willpower. It’s also quite easy to see a number of ways in which this could happen as an artifact of their methodology.
They later produced a second replication with the same subjects. They fail to find an effect, but they use “tendency to clear more lines at a time in tetris” as their measure of self-control, which as far as I know has never been used in another study and doesn’t seem very compelling to me. I don’t know why in god’s name they wouldn’t just replicate with any one of a dozen standard tests of willpower depletion. I would be curious if anyone has insight into this.
It looks like neither paper has any other citations, though they have seen a good bit of popular discussion. My best guess is that this is another example of it being easy for a crappy replication to fail to reproduce an effect regardless of whether it is real.
I remain agnostic about the original effect (at least relative to my disdain for this study). It looks to me like the evidence is reasonably good, and if true it certainly seems like an important fact. I’d be curious if anyone has thoughts on it.
ETA my best guess: willpower depletion is real, the perception of glucose (but not an artificial sweetener) really does help a lot with performance 10m later if in a depleted state, and recently agreeing to “I don’t think willpower depletion is real” causes people to put in much more effort when in a depleted state. The theory “people who believe in willpower depletion experience it more” does explain the latter observation, but the correlational studies seem to suffer badly from reverse causality, and the experimental studies seem to involve implausibly large effects given the posited mechanism, so I suspect that something else is going on (and a number of contenders do leap to mind).