Decision-Driven Regularization: A Blended Model for Predict-then-Optimize

Abstract

In contextual optimization, the decision-maker seeks optimal decisions to minimize a cost function, that varies based on observed features. This context is common in many business applications ranging from on-demand delivery and retail operations to portfolio optimization and inventory management. In this paper, we study the predict-then-optimize approach, which first learns how outcomes result from the features, and then selects optimal decisions based on these outcomes. We propose a principled manner to construct a regularization for forming the predictions, via the subsequent optimization problem of selecting decisions to minimize cost. We term our framework decision-driven regularization, which is a blended predict-then-optimize framework that carries both elements of predictive accuracy and decision quality, and is tractable. It also addresses ambiguity in the definition of the cost function, via a surrogate that depends on a new hyper-parameter. We additionally show that alternative perspectives for formulating the problem, namely robust optimization and regret minimization, lead to models that are equivalent to or can be naturally approximated to our proposed model. As a consequence, our framework generalizes models such as SPO+ in Elmachtoub and Grigas (2021). We present reasons for why improving predictive accuracy is insufficient in the contextual stochastic optimization setting, and can even be detrimental, to improving the quality of decisions. Our model is shown to be numerically superior to other benchmarks, such as OLS, SPO+, Lasso, and Ridge, in our synthetic studies.

Keywords: joint prediction and optimization, robust optimization, worst-case regret minimization, regularization, decision-making under uncertainty

JEL Classification: C44, C10