Interesting, hadn’t heard of this! Haven’t fully grasped the “No evidence for nudging after adjusting for publication bias” study yet, but at first glance it looks to me as if it is rather evidence for small effect sizes than for no effect at all? Generally, when people say “nudging doesn’t work”, this can mean a lot of things, from “there’s no effect at all” to “there often is an effect, but it’s not very large, and it’s not worth it to focus on this in policy debates”, to “it has a significant effect, but it will never solve a problem fully because it only affects the behavior of a minority of subjects”.
There’s also this article making some similar points, overall defending the effectiveness of nudging while also pushing for more nuance in the debate. They cite one very large study in particular that showed significant effects while avoiding publication bias (emphasis mine):
The study was unique because these organizations had provided access to the full universe of their trials—not just ones selected for publication. Across 165 trials testing 349 interventions, reaching more than 24 million people, the analysis shows a clear, positive effect from the interventions. On average, the projects produced an average improvement of 8.1 percent on a range of policy outcomes. The authors call this “sizable and highly statistically significant,” and point out that the studies had better statistical power than comparable academic studies. So real-world interventions do have an effect, independent of publication bias. (...) We can start to see the bigger problem here. We have a simplistic and binary “works” versus “does not work” debate. But this is based on lumping together a massive range of different things under the “nudge” label, and then attaching a single effect size to that label.
Personally I have a very strong prior that nudging must have an effect > 0 - it would just be extremely surprising to me if the effect of an intervention that clearly points in one direction would be exactly 0. This may however still be compatible with the effects in many cases being too small to be worth to put the spotlight on, and I suspect it just strongly depends on the individual case and intervention.
Personally I have a very strong prior that nudging must have an effect > 0 - it would just be extremely surprising to me if the effect of an intervention that clearly points in one direction would be exactly 0. This may however still be compatible with the effects in many cases being too small to be worth to put the spotlight on, and I suspect it just strongly depends on the individual case and intervention.
How to combine this with the fact that “the nudge” apparently doesn’t work https://phys.org/news/2022-08-nudge-theory-doesnt-evidence-future.html ?
Interesting, hadn’t heard of this! Haven’t fully grasped the “No evidence for nudging after adjusting for publication bias” study yet, but at first glance it looks to me as if it is rather evidence for small effect sizes than for no effect at all? Generally, when people say “nudging doesn’t work”, this can mean a lot of things, from “there’s no effect at all” to “there often is an effect, but it’s not very large, and it’s not worth it to focus on this in policy debates”, to “it has a significant effect, but it will never solve a problem fully because it only affects the behavior of a minority of subjects”.
There’s also this article making some similar points, overall defending the effectiveness of nudging while also pushing for more nuance in the debate. They cite one very large study in particular that showed significant effects while avoiding publication bias (emphasis mine):
Personally I have a very strong prior that nudging must have an effect > 0 - it would just be extremely surprising to me if the effect of an intervention that clearly points in one direction would be exactly 0. This may however still be compatible with the effects in many cases being too small to be worth to put the spotlight on, and I suspect it just strongly depends on the individual case and intervention.
Just to note I wrote a separate post focusing on pretty much that last point: