Assessing Nudge Scalability

96 Pages Posted: 20 Jan 2022 Last revised: 6 Jun 2023

See all articles by Silvia Saccardo

Silvia Saccardo

Carnegie Mellon University, Department of Social and Decision Sciences

Hengchen Dai

University of California, Los Angeles (UCLA) - Anderson School of Management

Maria Han

University of California, Los Angeles (UCLA) - UCLA Health System

Naveen Raja

University of California, Los Angeles (UCLA) - UCLA Health System

Sitaram Vangala

University of California, Los Angeles (UCLA) - Division of General Internal Medicine

Daniel Croymans

University of California, Los Angeles (UCLA) - UCLA Health System

Date Written: June 5, 2023

Abstract

Field experimentation and behavioral science have the potential to inform policy. Yet, many initially promising interventions show substantially lower efficacy at scale, reflecting the broader issue of the instability of scientific findings. We identify important factors that can explain variation in estimated intervention efficacy across evaluations and help policymakers better predict behavioral responses to interventions in their settings. We leverage data from (1) 123 randomized control trials (RCTs) involving over 20 million people that were conducted by either academics or a government agency to evaluate nudges, and (2) two RCTs (N=187,134 and 149,720) that we conducted to nudge COVID-19 vaccinations. Across those datasets, we find that nudges' estimated efficacy tends to be higher when outcomes are more narrowly (vs. broadly) defined and measured over a shorter (vs. longer) horizon. Nudges' impact is quite limited among individuals with low baseline motivation to engage in the target activity--a finding that is masked when only focusing on average effects. We further show that taking into account outcome breadth, outcome horizon, and a study population's baseline motivation can partially explain why nudges evaluated by academics show substantially larger effect sizes on average than nudges evaluated at scale by government agencies. Altogether, these results highlight that considering how intervention effectiveness is measured and who is nudged is critical to reconciling differences in effect sizes across evaluations and assessing the scalability of empirical findings.

Note:
Funding: Funding support for this research was provided by UCLA Health.

Declaration of Interests: The authors declare no competing interests. The authors did not receive financial or non-financial benefits from UCLA Health or speaking/consulting fees related to any of the interventions presented here.

Ethics Approval Statement: Our randomized control trials (RCTs) that promoted COVID-19 vaccine uptake were reviewed and approved by the Institutional Review Board (IRB) at the University of California, Los Angeles, which determined that a waiver of informed consent was appropriate.

Trial Registration: clinicaltrials.gov numbers: NCT04800965 and NCT04801524.

Keywords: Nudge, Scalability, Reproducibility, Randomized controlled trials

JEL Classification: C93, I12

Suggested Citation

Saccardo, Silvia and Dai, Hengchen and Han, Maria and Raja, Naveen and Vangala, Sitaram and Croymans, Daniel, Assessing Nudge Scalability (June 5, 2023). Available at SSRN: https://ssrn.com/abstract=3971192 or http://dx.doi.org/10.2139/ssrn.3971192

Silvia Saccardo

Carnegie Mellon University, Department of Social and Decision Sciences ( email )

Pittsburgh, PA 15213-3890
United States

Hengchen Dai (Contact Author)

University of California, Los Angeles (UCLA) - Anderson School of Management ( email )

110 Westwood Plaza
Los Angeles, CA 90095-1481
United States

Maria Han

University of California, Los Angeles (UCLA) - UCLA Health System ( email )

10833 Le Conte Avenue
17-165 CHS
Los Angeles, CA 90095-1730
United States

Naveen Raja

University of California, Los Angeles (UCLA) - UCLA Health System ( email )

10833 Le Conte Avenue
17-165 CHS
Los Angeles, CA 90095-1730
United States

Sitaram Vangala

University of California, Los Angeles (UCLA) - Division of General Internal Medicine ( email )

Los Angeles, CA
United States

Daniel Croymans

University of California, Los Angeles (UCLA) - UCLA Health System ( email )

10833 Le Conte Avenue
17-165 CHS
Los Angeles, CA 90095-1730
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
475
Abstract Views
2,248
Rank
113,108
PlumX Metrics