Characterizing Selection Bias Using Experimental Data

98 Pages Posted: 19 Nov 1998 Last revised: 26 Oct 2022

See all articles by James J. Heckman

James J. Heckman

University of Chicago - Department of Economics; National Bureau of Economic Research (NBER); American Bar Foundation; Institute for the Study of Labor (IZA); CESifo (Center for Economic Studies and Ifo Institute)

Hidehiko Ichimura

Graduate School of Economics, University of Tokyo

Jeffrey A. Smith

University of Wisconsin - Madison; National Bureau of Economic Research (NBER); Institute for the Study of Labor (IZA)

Petra Todd

University of Pennsylvania - Department of Economics; National Bureau of Economic Research (NBER); IZA Institute of Labor Economics

Date Written: August 1998

Abstract

This paper develops and applies semiparametric econometric methods to estimate the form of selection bias that arises from using nonexperimental comparison groups to evaluate social programs and to test the identifying assumptions that justify three widely-used classes of estimators and our extensions of them: (a) the method of matching; (b) the classical econometric selection model which represents the bias solely as a function of the probability of participation; and (c) the method of difference-in-differences. Using data from an experiment on a prototypical social program combined with unusually rich data from a nonexperimental comparison group, we reject the assumptions justifying matching and our extensions of that method but find evidence in support of the index-sufficient selection bias model and the assumptions that justify application of a conditional semiparametric version of the method of difference-in-difference. Fa comparable people and to appropriately weight participants and nonparticipants a sources of selection bias as conveniently measured. We present a rigorous defin bias and find that in our data it is a small component of conventially meausred it is still substantial when compared with experimentally-estimated program impa matching participants to comparison group members in the same labor market, givi same questionnaire, and making sure they have comparable characteristics substan the performance of any econometric program evaluation estimator. We show how t analysis to estimate the impact of treatment on the treated using ordinary obser

Suggested Citation

Heckman, James J. and Ichimura, Hidehiko and Smith, Jeffrey Andrew and Todd, Petra, Characterizing Selection Bias Using Experimental Data (August 1998). NBER Working Paper No. w6699, Available at SSRN: https://ssrn.com/abstract=122609

James J. Heckman (Contact Author)

University of Chicago - Department of Economics ( email )

1126 East 59th Street
Chicago, IL 60637
United States
773-702-0634 (Phone)
773-702-8490 (Fax)

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

American Bar Foundation

750 N. Lake Shore Drive
Chicago, IL 60611
United States

Institute for the Study of Labor (IZA)

P.O. Box 7240
Bonn, D-53072
Germany

CESifo (Center for Economic Studies and Ifo Institute)

Poschinger Str. 5
Munich, DE-81679
Germany

Hidehiko Ichimura

Graduate School of Economics, University of Tokyo ( email )

7-3-1 Hongo Bunkyo-ku
Tokyo 113-0033
Japan

Jeffrey Andrew Smith

University of Wisconsin - Madison ( email )

716 Langdon Street
Madison, WI 53706-1481
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Institute for the Study of Labor (IZA)

P.O. Box 7240
Bonn, D-53072
Germany

Petra Todd

University of Pennsylvania - Department of Economics ( email )

Ronald O. Perelman Center for Political Science
133 South 36th Street
Philadelphia, PA 19104-6297
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

IZA Institute of Labor Economics

P.O. Box 7240
Bonn, D-53072
Germany

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
215
Abstract Views
18,746
Rank
255,999
PlumX Metrics