Are Survey Weights Needed? A Review of Diagnostic Tests in Regression Analysis

Posted: 3 Jun 2016

See all articles by Kenneth Bollen

Kenneth Bollen

University of North Carolina (UNC) at Chapel Hill

Paul Biemer

University of North Carolina (UNC) at Chapel Hill - Odum Institute for Research in Social Science

Alan Karr

RTI International

Stephen Tueller

RTI International - Headquarters - Research Triangle Park

Marcus E. Berzofsky

RTI International - Headquarters - Research Triangle Park

Date Written: June 2016

Abstract

Researchers apply sampling weights to take account of unequal sample selection probabilities and to frame coverage errors and nonresponses. If researchers do not weight when appropriate, they risk having biased estimates. Alternatively, when they unnecessarily apply weights, they can create an inefficient estimator without reducing bias. Yet in practice researchers rarely test the necessity of weighting and are sometimes guided more by the current practice in their field than by scientific evidence. In addition, statistical tests for weighting are not widely known or available. This article reviews empirical tests to determine whether weighted analyses are justified. We focus on regression models, though the review's implications extend beyond regression. We find that nearly all weighting tests fall into two categories: difference in coefficients tests and weight association tests. We describe the distinguishing features of each category, present their properties, and explain the close relationship between them. We review the simulation evidence on their sampling properties in finite samples. Finally, we highlight the unanswered theoretical and practical questions that surround these tests and that deserve further research.

Suggested Citation

Bollen, Kenneth and Biemer, Paul and Karr, Alan and Tueller, Stephen and Berzofsky, Marcus E., Are Survey Weights Needed? A Review of Diagnostic Tests in Regression Analysis (June 2016). Annual Review of Statistics and Its Application, Vol. 3, Issue 1, pp. 375-392, 2016, Available at SSRN: https://ssrn.com/abstract=2789503 or http://dx.doi.org/10.1146/annurev-statistics-011516-012958

Kenneth Bollen (Contact Author)

University of North Carolina (UNC) at Chapel Hill ( email )

102 Ridge Road
NC 27514

Paul Biemer

University of North Carolina (UNC) at Chapel Hill - Odum Institute for Research in Social Science ( email )

Chapel HIll, NC 27599
United States

Alan Karr

RTI International ( email )

PO Box 12194
Research Triangle Park, 27709
United States

Stephen Tueller

RTI International - Headquarters - Research Triangle Park

3040 Cornwallis Road
P.O. Box 12194
Research Triangle Park, NC 27709
United States

Marcus E. Berzofsky

RTI International - Headquarters - Research Triangle Park

3040 Cornwallis Road
P.O. Box 12194
Research Triangle Park, NC 27709
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
249
PlumX Metrics