The Costs of Simplicity: Why Multilevel Models May Benefit from Accounting for Cross-Cluster Differences in the Effects of Controls

89 Pages Posted: 16 Dec 2015 Last revised: 28 Jan 2017

See all articles by Jan Heisig

Jan Heisig

WZB Berlin Social Science Center

Merlin Schaeffer

University of Cologne

Johannes Giesecke

Humboldt University of Berlin

Date Written: January 27, 2017

Abstract

Context effects, where a characteristic of an upper-level unit or cluster (e.g., a country) affects outcomes and relationships at a lower level (e.g., that of the individual), are a primary object of sociological inquiry. In recent years, sociologists have increasingly analyzed such effects using quantitative multilevel modeling. Our review of multilevel studies in leading sociology journals shows that most assume the effects of lower-level control variables to be invariant across clusters, an assumption that is often implausible. Comparing mixed effects (random intercept and slope) models, cluster-robust pooled OLS, and two-step approaches, we find that erroneously assuming invariant coefficients reduces the precision of estimated context effects. Semi-formal reasoning and Monte Carlo simulations indicate that the loss of precision is larg- est when there is pronounced cross-cluster heterogeneity in the magnitude of coefficients, when there are marked compositional differences among clusters, and when the number of clusters is small. While these findings suggest that practitioners should fit more flexible mod- els, exemplary analyses of European Social Survey data indicate that maximally flexible mixed effects models do not perform well in real-life settings. We discuss the need to develop principled procedures for balancing parsimony and flexibility and demonstrate the encourag- ing performance of one prominent approach.

Keywords: Hierarchical Regression, Multilevel Modeling, Cluster-Robust Standard Errors, Country Comparison, Camporative Research

JEL Classification: C15, C21, C51, C52

Suggested Citation

Heisig, Jan and Schaeffer, Merlin and Giesecke, Johannes, The Costs of Simplicity: Why Multilevel Models May Benefit from Accounting for Cross-Cluster Differences in the Effects of Controls (January 27, 2017). Available at SSRN: https://ssrn.com/abstract=2703431 or http://dx.doi.org/10.2139/ssrn.2703431

Jan Heisig (Contact Author)

WZB Berlin Social Science Center ( email )

Reichpietschufer 50
D-10785 Berlin, 10785
Germany

Merlin Schaeffer

University of Cologne ( email )

Albertus-Magnus-Platz
Cologne, 50923
Germany

Johannes Giesecke

Humboldt University of Berlin ( email )

Unter den Linden 6
Berlin, AK Berlin 10099
Germany

Register to save articles to
your library

Register

Paper statistics

Downloads
395
rank
72,070
Abstract Views
1,743
PlumX Metrics