Principles for Examining Predictive Validity: The Case of Information Systems Spending Forecasts

Information Systems Research, Vol. 5, pp. 170-179, June 1994

10 Pages Posted: 21 Jul 2008

See all articles by J. Scott Armstrong

J. Scott Armstrong

University of Pennsylvania - Marketing Department

Fred Collopy

Case Western Reserve University - Department of Information Systems

Monica Adya

Marquette University - College of Business Administration

Date Written: July 21, 2008

Abstract

Research over two decades has advanced the knowledge of how to assess predictive validity. We believe this has value to information systems (IS) researchers. To demonstrate, we used a widely cited study of IS spending. In that study, price-adjusted diffusion models were proposed to explain and to forecast aggregate U.S. information systems spending. That study concluded that such models would produce more accurate forecasts than would simple linear trend extrapolation. However, one can argue that the validation procedure provided an advantage to the diffusion models. We reexamined the results using an alternative validation procedure based on three principles extracted from forecasting research: (1) use ex ante (out-of-sample) performance rather than the fit to the historical data, (2) use well-accepted models as a basis for comparison, and (3) use an adequate sample of forecasts. Validation using this alternative procedure did confirm the importance of the price-adjustment, but simple trend extrapolations were found to be more accurate than the price-adjusted diffusion models.

Suggested Citation

Armstrong, J. Scott and Collopy, Fred and Adya, Monica, Principles for Examining Predictive Validity: The Case of Information Systems Spending Forecasts (July 21, 2008). Information Systems Research, Vol. 5, pp. 170-179, June 1994. Available at SSRN: https://ssrn.com/abstract=1164782

J. Scott Armstrong (Contact Author)

University of Pennsylvania - Marketing Department ( email )

700 Jon M. Huntsman Hall
3730 Walnut Street
Philadelphia, PA 19104-6340
United States
215-898-5087 (Phone)
215-898-2534 (Fax)

HOME PAGE: http://marketing.wharton.upenn.edu/people/faculty/armstrong.cfm

Fred Collopy

Case Western Reserve University - Department of Information Systems ( email )

10900 Euclid Ave.
Cleveland, OH 44106-7235
United States

Monica Adya

Marquette University - College of Business Administration ( email )

P.O. Box 1881
Milwaukee, WI 53201-1881
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
29
Abstract Views
389
PlumX Metrics