Elements of External Validity: Framework, Design, and Analysis

85 Pages Posted:

See all articles by Naoki Egami

Naoki Egami

Columbia University

Erin Hartman

University of California, Los Angeles (UCLA)

Date Written: June 30, 2020

Abstract

External validity of randomized experiments is a focus of long-standing debates in the social sciences. While the issue has been extensively studied at the conceptual level, in practice, few empirical studies have explicit analysis aimed towards externally valid inferences. In this article, we make three contributions to improve empirical approaches for external validity. First, we propose a formal framework that encompasses four dimensions of external validity; X-, T-, Y -, and C-validity (units, treatments, outcomes, and contexts). The proposed framework synthesizes diverse external validity concerns that arise in practice. We then distinguish two goals of generalization. To conduct effect-generalization — generalizing the magnitude of causal effects, we introduce three estimators of the target population causal effects. For sign-generalization — assessing whether the direction of causal effects is generalizable, we propose a novel multiple-testing procedure under weaker assumptions. We illustrate our methods through three applications covering field, survey, and lab experiments.

Keywords: Causal inference, External validity, Generalization, Randomized Experiment

Suggested Citation

Egami, Naoki and Hartman, Erin, Elements of External Validity: Framework, Design, and Analysis (June 30, 2020). Available at SSRN: https://ssrn.com/abstract=3775158

Naoki Egami (Contact Author)

Columbia University ( email )

7th Floor, International Affairs Bldg.
420 W. 118th Street
New York, NY 10027
United States

HOME PAGE: http://https://naokiegami.com

Erin Hartman

University of California, Los Angeles (UCLA) ( email )

405 Hilgard Avenue
Box 951361
Los Angeles, CA 90095
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
20
Abstract Views
89
PlumX Metrics