Predictive Analytics in Changing Environments: Trade-Offs in Response to Changes in Data Patterns

30 Pages Posted: 8 Nov 2021

See all articles by Jiaxu Peng

Jiaxu Peng

Central University of Finance and Economics - Department of Accounting Informatization System

Jungpil Hahn

National University of Singapore (NUS) - Department of Information Systems and Analytics

Dandan Qiao

National University of Singapore

Date Written: August 23, 2021

Abstract

Despite advances in analytics techniques and access to abundant data, making accurate predictions is still challenging due to changing data environments as we lack sufficient information to adjust prediction models in a timely manner. Data analysts face two trade-offs – 1) whether or not they should use the large but potentially less relevant historical source data, and 2) whether they should adjust the prediction model immediately when change in data is suspected or later when more relevant post-change source data becomes available. These trade-offs are related to the fundamental bias-variance and exploration-exploitation trade- offs. To aid data analysts’ deal with these trade-offs, we develop theoretical insights by adopting a sample selection perspective to represent changes in data patterns. Based on the sample selection model, we propose a transfer learning framework that leverages the large historical source data. The framework allows us to theoretically study the two fundamental trade-offs. Our analysis shows that using historical source data offers relatively larger benefits when the prediction model has higher complexity, and when the extent of change is not too large. Moreover, the benefit also varies with the scarcity of post-change source data thus with the timing of adjusting the prediction model. Our theoretical analysis shows that the decisions on the two trade-offs interact with each other. Our analysis is further corroborated by Monte Carlo simulations in a data change detection and model adjustment context. Overall, this study provides theoretical insights and practical guidelines for the application of predictive analytics in changing data environments.

Keywords: changing data environments, predictive analytics, transfer learning, sample selection, data heterogeneity, statistical learning

Suggested Citation

Peng, Jiaxu and Hahn, Jungpil and Qiao, Dandan, Predictive Analytics in Changing Environments: Trade-Offs in Response to Changes in Data Patterns (August 23, 2021). Available at SSRN: https://ssrn.com/abstract=3956423 or http://dx.doi.org/10.2139/ssrn.3956423

Jiaxu Peng

Central University of Finance and Economics - Department of Accounting Informatization System ( email )

39 South College Road
Haidian District,Beijing
Beijing, 100081
China

Jungpil Hahn (Contact Author)

National University of Singapore (NUS) - Department of Information Systems and Analytics ( email )

Singapore

Dandan Qiao

National University of Singapore ( email )

13 computing drive
Singapore, 117417
Singapore

Do you want regular updates from SSRN on Twitter?

Paper statistics

Downloads
18
Abstract Views
162
PlumX Metrics