On the Strength of Relaxations of Weakly Coupled Stochastic Dynamic Programs

46 Pages Posted: 23 May 2021 Last revised: 7 Feb 2022

See all articles by David B. Brown

David B. Brown

Duke University - Decision Sciences

Jingwei Zhang

Duke University - Decision Sciences

Date Written: May 19, 2021

Abstract

Many stochastic dynamic programs (DPs) have a weakly coupled structure in that a set of linking constraints in each period couple an otherwise independent collection of subproblems. Two widely studied approximations of such problems are approximate linear programs (ALP), which involve optimizing value function approximations that additively separate across subproblems, and Lagrangian relaxations, which involve relaxing the linking constraints. It is well-known that both of these approximations provide upper bounds on the optimal value function in all states and that the ALP provides a tighter upper bound in the initial state. The purpose of this short paper is to provide theoretical justification for the fact that these upper bounds are often close if not identical. We show:
(i) For any weakly coupled DP, the difference between these two upper bounds --- the relaxation gap --- is bounded from above in terms of the integrality gap of the separation problems associated with the ALP;
(ii) If subproblem rewards are uniformly bounded and some broadly applicable conditions on the linking constraints hold, the relaxation gap is bounded from above by a constant that is independent of the number of subproblems; and
(iii) When the linking constraints are independent of subproblem states and have a unimodular structure, the relaxation gap equals zero.
The conditions for (iii) hold in several widely studied problems: generalizations of restless bandit problems, online stochastic matching problems, network revenue management problems, and price-directed control of relocating resources. These findings generalize and unify existing results.

Keywords: Weakly coupled stochastic dynamic programs, approximate dynamic programming, Lagrangian relaxations, restless bandits, network revenue management

JEL Classification: C61

Suggested Citation

Brown, David B. and Zhang, Jingwei, On the Strength of Relaxations of Weakly Coupled Stochastic Dynamic Programs (May 19, 2021). Available at SSRN: https://ssrn.com/abstract=3849573 or http://dx.doi.org/10.2139/ssrn.3849573

David B. Brown (Contact Author)

Duke University - Decision Sciences ( email )

Durham, NC 27708-0120
United States

Jingwei Zhang

Duke University - Decision Sciences ( email )

Durham, NC 27708-0120
United States

Do you want regular updates from SSRN on Twitter?

Paper statistics

Downloads
138
Abstract Views
727
rank
282,541
PlumX Metrics