Abstract

http://ssrn.com/abstract=1336700
 
 

References (59)



 


 



Too Big to Fail: Larger Samples and False Discoveries


Mingfeng Lin


University of Arizona - Eller College of Management

Henry C. Lucas


University of Maryland - Robert H. Smith School of Business

Galit Shmueli


Indian School of Business

June 15, 2011

Robert H. Smith School Research Paper No. RHS 06-068

Abstract:     
The Internet presents great opportunities for research about information technology, allowing IS researchers to collect very large and rich datasets. It is common to see research papers with tens or even hundreds of thousands of data points, especially when reading about electronic commerce. Large samples are better than smaller samples in that they provide greater statistical power and produce more precise estimates. However, statistical inference using p-values does not scale up to large samples and often leads to erroneous conclusions. We find evidence of an over-reliance on p-values in large sample IS studies in top IS journals and conferences. In this commentary, we focus on interpreting effects of individual independent variables on a dependent variable in regression-type models. We discuss how p-values become deflated with a large sample and illustrate this deflation in analyzing data from over 340,000 digital camera auctions on eBay. The commentary recommends that IS researchers be more conservative in interpreting statistical significance in large sample studies, and instead, interpret results in terms of practical significance. In particular, we suggest that authors of large-sample IS studies report and discuss confidence intervals for independent variables of interest rather than coefficient signs and p-values. We also suggest taking advantage of a large dataset for examining how coefficients and p-values change as sample size increases, and for estimating models on multiple subsamples to further test robustness.

Number of Pages in PDF File: 37

Keywords: large samples, p-values, statistical significance, practical significance

working papers series





Download This Paper

Date posted: February 3, 2009 ; Last revised: July 25, 2011

Suggested Citation

Lin, Mingfeng and Lucas, Henry C. and Shmueli, Galit, Too Big to Fail: Larger Samples and False Discoveries (June 15, 2011). Robert H. Smith School Research Paper No. RHS 06-068. Available at SSRN: http://ssrn.com/abstract=1336700 or http://dx.doi.org/10.2139/ssrn.1336700

Contact Information

Mingfeng Lin (Contact Author)
University of Arizona - Eller College of Management ( email )
Tucson, AZ 85721
United States
Henry C. Lucas
University of Maryland - Robert H. Smith School of Business ( email )
College Park, MD 20742-1815
United States
Galit Shmueli
Indian School of Business ( email )
Hyderabad, Gachibowli 500 032
India
HOME PAGE: http://galitshmueli.com

Feedback to SSRN


Paper statistics
Abstract Views: 2,210
Downloads: 443
Download Rank: 36,608
References:  59

© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollo5 in 0.625 seconds