Evaluating the Substantive Significance of Linear Fixed Effects Regression Results

31 Pages Posted: 2 Sep 2013 Last revised: 5 Sep 2013

See all articles by Jonathan Mummolo

Jonathan Mummolo

Princeton University

Erik Peterson

Texas A&M University - Department of Political Science

Date Written: September 2, 2013

Abstract

The counterfactuals researchers commonly use to assess the substantive significance of linear fixed effects regression results do not account for the manner in which these models are estimated. Importantly, including fixed effects in a regression means that the model is estimated based on the often narrow within-unit distribution of an independent variable. Despite this, counterfactuals are often motivated with features of an independent variable's overall distribution (e.g., its range or standard deviation). Using simulated data and two case studies, we show this approach has two consequences. First, it inflates the substantive significance of any variables assessed in this way. Second, these counterfactuals are unreliable and exhibit a high degree of model dependence. We recommend instead that researchers assess the substantive significance of fixed effects regression results with counterfactuals based on the within-unit distribution of an independent variable. We provide an R function to implement this recommendation.

Keywords: fixed effects, substantive significance, reliability, linear regression

Suggested Citation

Mummolo, Jonathan and Peterson, Erik, Evaluating the Substantive Significance of Linear Fixed Effects Regression Results (September 2, 2013). Available at SSRN: https://ssrn.com/abstract=2319506 or http://dx.doi.org/10.2139/ssrn.2319506

Jonathan Mummolo (Contact Author)

Princeton University ( email )

22 Chambers Street
Princeton, NJ 08544-0708
United States

Erik Peterson

Texas A&M University - Department of Political Science ( email )

College Station, TX 77843-4353
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
47
Abstract Views
822
PlumX Metrics