Optimize-via-Predict: Realizing out-of-sample optimality in data-driven optimization

42 Pages Posted: 25 Sep 2023 Last revised: 22 Dec 2023

See all articles by Gar Goei Loke

Gar Goei Loke

Durham University Business School

Taozeng Zhu

Dongbei University of Finance and Economics

Ruiting Zuo

Fintech Thrust, the Society Hub, Hong Kong University of Science and Technology (GZ)

Date Written: September 4, 2023

Abstract

In this paper, we propose a novel methodology that achieves almost zero regret in data-driven optimization over a localized region of possible truths. We consider the data-driven optimization setting wherein the uncertainty lies in a parametric family and the decision-maker is not privy to the true parameter, but possesses a historical data set. We define a prescriptive solution as a decision rule mapping such a data set to decisions, and its finite-sample out-of-sample performance as the average over data sets under the sampling distribution. The goal is an optimal prescriptive solution. We prove that there cannot exist prescriptive solutions that are unconditionally generalizable over all true parameters. Thus, we propose the definition of local optimality that averages the prescriptive solution's performance over a neighbourhood of parameters. We prove sufficient conditions for local optimality, which reduces to functions of the sufficient statistic of the parameter. We present an optimization problem that yields a locally optimal solution and can be solved efficiently. We also state sufficient conditions for the existence and uniqueness of this solution. Next, we prove that there is specificity-sensitivity trade-off in terms of the size of the neighbourhood considered. Finally, we illustrate our model on the newsvendor model to find strong performance when compared against alternatives in the literature. There are potential extensions to contextual optimization and Bayesian optimization.

Keywords: Data-driven optimization, Prescriptive analytics, Sufficient statistics, Robust optimization, Stochastic optimization, Finite-sample optimality

JEL Classification: C10, C44, C60

Suggested Citation

Loke, Gar Goei and Zhu, Taozeng and Zuo, Ruiting, Optimize-via-Predict: Realizing out-of-sample optimality in data-driven optimization (September 4, 2023). Available at SSRN: https://ssrn.com/abstract=4561006 or http://dx.doi.org/10.2139/ssrn.4561006

Gar Goei Loke

Durham University Business School ( email )

Mill Hill Lane
Durham, DH1 3LB
United Kingdom

Taozeng Zhu

Dongbei University of Finance and Economics ( email )

Dalian
China

Ruiting Zuo (Contact Author)

Fintech Thrust, the Society Hub, Hong Kong University of Science and Technology (GZ) ( email )

+86 18256944842 (Phone)

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
153
Abstract Views
615
Rank
360,790
PlumX Metrics