Scaling Nudges: Who Moves and How

Abstract

Field experimentation and behavioral science have the potential to inform policy. Yet, many initially promising interventions show substantially lower efficacy at scale (e.g., List et al., 2024), reflecting the broader issue of the instability of experimental findings across contexts. We identify two important factors that can explain variation in estimated intervention efficacy across evaluations. We analyze data from (1) 123 randomized controlled trials (RCTs) from the nudge literature, covering over 20 million people, and (2) two RCTs (N = 187,134 and 149,720)  encouraging COVID-19 vaccinations.  We find that intervention efficacy tends to be smaller (1) among individuals with low (vs. moderately high) baseline propensity to engage in the target activity and (2) when outcome measures are defined more broadly (vs. narrowly), as broader  measures better account for substitution effects and more accurately reflect net behavior change.  These findings help reconcile discrepancies in reported effect sizes - including the gap between academic- and government-led evaluations (Della Vigna & Linos, 2022) - and offer theoretical insight and actionable guidance  for selecting and scaling interventions.

Note:
Funding: Funding support for this research was provided by UCLA Health.

Declaration of Interests: The authors declare no competing interests. The authors did not receive financial or non-financial benefits from UCLA Health or speaking/consulting fees related to any of the interventions presented here.

Ethics Approval Statement: Our randomized control trials (RCTs) that promoted COVID-19 vaccine uptake were reviewed and approved by the Institutional Review Board (IRB) at the University of California, Los Angeles, which determined that a waiver of informed consent was appropriate.

Trial Registration: clinicaltrials.gov numbers: NCT04800965 and NCT04801524.

Keywords: Nudge, Scalability, Reproducibility, Randomized controlled trials

JEL Classification: C93, I12