Measurement Error in Performance Studies of Health Information Technology: Lessons from the Management Literature

Applied Clinical Informatics, Vol. 3, No. 2, pp. 210-220, 2012

11 Pages Posted: 21 Jun 2012 Last revised: 10 Jun 2013

See all articles by Adam Seth Litwin

Adam Seth Litwin

Cornell University; Cornell University, Ithaca, New York

Ariel C. Avgar

University of Illinois at Urbana-Champaign, School of Labor & Employment Relations and College of Medicine

Peter Pronovost

Johns Hopkins University - Johns Hopkins Medical Institutions

Date Written: June 1, 2012

Abstract

Just as researchers and clinicians struggle to pin down the benefits attendant to health information technology (IT), management scholars have long labored to identify the performance effects arising from new technologies and from other organizational innovations, namely the reorganization of work and the devolution of decision-making authority. This paper applies lessons from that literature to theorize the likely sources of measurement error that yield the weak statistical relationship between measures of health IT and various performance outcomes. In so doing, it complements the evaluation literature’s more conceptual examination of health IT’s limited performance impact. The paper focuses on seven issues, in particular, that likely bias downward the estimated performance effects of health IT. They are 1.) negative self-selection, 2.) omitted or unobserved variables, 3.) mismeasured contextual variables, 4.) mismeasured health IT variables, 5.) lack of attention to the specific stage of the adoption-to-use continuum being examined, 6.) too short of a time horizon, and 7.) inappropriate units-of-analysis. The authors offer ways to counter these challenges. Looking forward more broadly, they suggest that researchers take an organizationally-grounded approach that privileges internal validity over generalizability. This focus on statistical and empirical issues in health IT-performance studies should be complemented by a focus on theoretical issues, in particular, the ways that health IT creates value and apportions it to various stakeholders.

Keywords: health information technology (IT), electronic health records (EHRs), research methods, organizational behavior

Suggested Citation

Litwin, Adam Seth and Avgar, Ariel C. and Pronovost, Peter, Measurement Error in Performance Studies of Health Information Technology: Lessons from the Management Literature (June 1, 2012). Applied Clinical Informatics, Vol. 3, No. 2, pp. 210-220, 2012, Available at SSRN: https://ssrn.com/abstract=2089081

Adam Seth Litwin (Contact Author)

Cornell University ( email )

ILR School
363 Ives Hall
Ithaca, NY 14853-3901
United States
(607) 255-7326 (Phone)

Cornell University, Ithaca, New York ( email )

New York
United States

HOME PAGE: http://https://www.ilr.cornell.edu/people/adam-seth-litwin

Ariel C. Avgar

University of Illinois at Urbana-Champaign, School of Labor & Employment Relations and College of Medicine

504 East Armory Avenue
Champaign, IL 61820-6297
United States

Peter Pronovost

Johns Hopkins University - Johns Hopkins Medical Institutions ( email )

550 N. Broadway
Baltimore, MD 21205
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
72
Abstract Views
1,095
Rank
693,070
PlumX Metrics