The Response Rate Test: Nonresponse Bias and the Future of Survey Research in Criminology and Criminal Justice

11 Pages Posted: 24 Jan 2018

See all articles by Justin Pickett

Justin Pickett

School of Criminal Justice, University at Albany, SUNY

Frank Cullen

University of Cincinnati

Shawn D. Bushway

University at Albany; RAND Corporation

Ted Chiricos

Florida State University

Geoffrey Alpert

University of South Carolina

Date Written: January 16, 2018

Abstract

There is a disciplinary assumption in our field that surveys with low response rates produce biased estimates, which leads to the use of simple rules for judging the quality of survey data. Surveys with “low” response rates fail this “response rate test” and become difficult to publish. Most of our research methods texts list these rules: e.g., “A response rate below 60% is a disaster, and even a 70% response rate is not much more than minimally acceptable”. Editors embrace this view, and often reject out of hand any study failing to reach this conventional standard. We argue that our field’s use of response rate rules in evaluating scholarship is based more on disciplinary custom than on survey science. In this paper, we describe the long-term downward trend in response rates and address confusion about nonresponse bias and its relation to response rates. Using Groves and Peytcheva’s (2008) meta-analytic data, we present evidence about the magnitude of the estimate- and study-level relationships between response rates and two different measures of nonresponse bias in univariate estimates. We then discuss several consequences of using the “response rate test” to judge data quality.

Keywords: Response rates, nonresponse bias, survey research, criminology, criminal justice

Suggested Citation

Pickett, Justin and Cullen, Frank and Bushway, Shawn D. and Chiricos, Ted and Alpert, Geoffrey, The Response Rate Test: Nonresponse Bias and the Future of Survey Research in Criminology and Criminal Justice (January 16, 2018). Available at SSRN: https://ssrn.com/abstract=3103018 or http://dx.doi.org/10.2139/ssrn.3103018

Justin Pickett (Contact Author)

School of Criminal Justice, University at Albany, SUNY ( email )

135 Western Avenue
Albany, NY 12222
United States
803-215-8954 (Phone)

Frank Cullen

University of Cincinnati ( email )

Cincinnati, OH 45221-0389
United States

Shawn D. Bushway

University at Albany ( email )

324 Milne Hall
Albany, NY 12222
United States

RAND Corporation ( email )

1776 Main Street
P.O. Box 2138
Santa Monica, CA 90407-2138
United States

Ted Chiricos

Florida State University ( email )

Tallahasse, FL 32306
United States

Geoffrey Alpert

University of South Carolina ( email )

701 Main Street
Columbia, SC 29208
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
743
Abstract Views
2,938
Rank
63,111
PlumX Metrics