Abstract

http://ssrn.com/abstract=288970
 
 

References (14)



 
 

Citations (714)



 


 



How Much Should We Trust Differences-in-Differences Estimates?


Marianne Bertrand


University of Chicago - Booth School of Business; National Bureau of Economic Research (NBER); Centre for Economic Policy Research (CEPR)

Esther Duflo


Massachusetts Institute of Technology (MIT) - Department of Economics; Abdul Latif Jameel Poverty Action Lab (J-PAL); National Bureau of Economic Research (NBER); Centre for Economic Policy Research (CEPR); Bureau for Research and Economic Analysis of Development (BREAD)

Sendhil Mullainathan


Harvard University - Department of Economics; National Bureau of Economic Research (NBER)

October 2001

MIT Department of Economics Working Paper No. 01-34

Abstract:     
Most Difference-in-Difference (DD) papers rely on many years of data and focus on serially correlated outcomes. Yet almost all these papers ignore the bias in the estimated standard errors that serial correlation introduces. This is especially troubling because the independent variable of interest in DD estimation (e.g., the passage of law) is itself very serially correlated, which will exacerbate the bias in standard errors. To illustrate the severity of this issue, we randomly generate placebo laws in state-level data on female wages from the Current Population Survey. For each law, we use OLS to compute the DD estimate of its "effect" as well as the standard error for this estimate. The standard errors are severely biased: with about 20 years of data, DD estimation finds an "effect" significant at the 5% level of up to 45% of the placebo laws.

Two very simple techniques can solve this problem for large sample sizes. The first technique consists in collapsing the data and ignoring the time-series variation altogether; the second technique is to estimate standard errors while allowing for an arbitrary covariance structure between time periods. We also suggest a third technique, based on randomization inference testing methods, which works well irrespective of sample size. This technique uses the empirical distribution of estimated effects for placebo laws to form the test distribution.

Number of Pages in PDF File: 38

Keywords: serial correlation; estimated standard errors; placebo laws, state-level female wages, randomization inference testing.

JEL Classification: C10, C13, E24, K39

working papers series


Download This Paper

Date posted: October 30, 2001  

Suggested Citation

Bertrand, Marianne and Duflo, Esther and Mullainathan, Sendhil, How Much Should We Trust Differences-in-Differences Estimates? (October 2001). MIT Department of Economics Working Paper No. 01-34. Available at SSRN: http://ssrn.com/abstract=288970 or http://dx.doi.org/10.2139/ssrn.288970

Contact Information

Marianne Bertrand
University of Chicago - Booth School of Business ( email )
5807 S. Woodlawn Avenue
Chicago, IL 60637
United States
773-834-5943 (Phone)
HOME PAGE: http://gsbwww.uchicago.edu/fac/marianne.bertrand/vita/cv_0604.pdf
National Bureau of Economic Research (NBER) ( email )
1050 Massachusetts Avenue
Cambridge, MA 02138
United States
617-588-0341 (Phone)
617-876-2742 (Fax)
Centre for Economic Policy Research (CEPR)
77 Bastwick Street
London, EC1V 3PZ
United Kingdom
Esther Duflo
Massachusetts Institute of Technology (MIT) - Department of Economics ( email )
50 Memorial Drive
Room E52-252G
Cambridge, MA 02142
United States
617-258-7013 (Phone)
617-253-6915 (Fax)
Abdul Latif Jameel Poverty Action Lab (J-PAL) ( email )
Cambridge, MA
United States
HOME PAGE: http://www.povertyactionlab.org/
National Bureau of Economic Research (NBER)
1050 Massachusetts Avenue
Cambridge, MA 02138
United States
Centre for Economic Policy Research (CEPR)
77 Bastwick Street
London, EC1V 3PZ
United Kingdom
Bureau for Research and Economic Analysis of Development (BREAD) ( email )
Duke University
Durham, NC 90097
United States
Sendhil Mullainathan (Contact Author)
Harvard University - Department of Economics ( email )
Littauer Center
Cambridge, MA 02138
United States
617-496-2720 (Phone)
617-495-7730 (Fax)
National Bureau of Economic Research (NBER) ( email )
1050 Massachusetts Avenue
Cambridge, MA 02138
United States
617-588-1473 (Phone)
617-876-2742 (Fax)
Feedback to SSRN


Paper statistics
Abstract Views: 3,995
Downloads: 866
Download Rank: 12,133
References:  14
Citations:  714

© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollo8 in 0.265 seconds