Are Surveys of Experts Unbiased? Evidence from College Football Rankings

21 Pages Posted: 6 Oct 2012

See all articles by Justin M. Ross

Justin M. Ross

Indiana University - School of Public & Environmental Affairs

Sarah Larson

University of Central Florida

Chad Wall

Virginia Commonwealth University (VCU)

Multiple version iconThere are 2 versions of this paper

Date Written: October 2012

Abstract

Policymakers are frequently interested in soliciting unbiased information regarding alternative policies, and expert surveys can be influential. As ranking policies is an often subjective process, there is always the concern of bias, both intentional and not. Expert bias is difficult to discern in the policy world, but surveys of expert opinion are compiled and “tested” for accuracy weekly in college football, allowing for hypothesis testing. Although previous research has used college football rankings to determine the ability of surveys to incorporate relevant information, this article examines the Associated Press and American Football Coaches' Association rankings for evidence of systematic bias. Specifically, more than 1,300 games from the 2003 to 2008 regular seasons are tested for factors that are systematically correlated with upsets. Both polls predict the winner nearly 80% of the time, and although there is evidence of systematic conference bias, correcting the rankings would only improve the accuracy of the polls by about 1%. There is no evidence of a bias favoring “big market” teams, nor teams that have strong journalism programs or are from the East Coast.

JEL Classification: D4, L8

Suggested Citation

Ross, Justin M. and Larson, Sarah and Wall, Chad, Are Surveys of Experts Unbiased? Evidence from College Football Rankings (October 2012). Contemporary Economic Policy, Vol. 30, Issue 4, pp. 502-522, 2012, Available at SSRN: https://ssrn.com/abstract=2157835 or http://dx.doi.org/10.1111/j.1465-7287.2011.00275.x

Justin M. Ross (Contact Author)

Indiana University - School of Public & Environmental Affairs ( email )

1315 East Tenth Street
Bloomington, IN 47405
United States

Sarah Larson

University of Central Florida ( email )

4364 Scorpius Street
HPA II, Room 238
Orlando, FL 32816-1395
United States
407-823-5839 (Phone)

HOME PAGE: http://www.larsonse.com

Chad Wall

Virginia Commonwealth University (VCU) ( email )

1015 Floyd Avenue
Richmond, VA 23284
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
1
Abstract Views
329
PlumX Metrics