Fluid Policies, Reoptimization, and Performance Guarantees in Dynamic Resource Allocation

Abstract

Many sequential decision problems involve deciding how to allocate shared resources across a set of independent systems at each point in time. A classic example is the restless bandit problem, in which a budget constraint limits the selection of arms at each point in time. Fluid relaxations provide a natural approximation technique for this broad class of problems. A recent stream of research has established strong performance guarantees for feasible policies based on fluid relaxations. In this paper, we generalize and improve these recent performance results. First, we show that a feasible fluid policy that chooses actions using a reoptimized fluid value function achieves performance within O(\sqrt{N}) of optimal, where N is the number of subproblems. This result holds for a general class of dynamic resource allocation problems with heterogeneous subproblems and multiple shared resource constraints. The proof technique is novel and improves upon results in the literature. In particular, the scaling of the constants with time implies a similar result in the infinite horizon setting. Second, for problems with a single shared resource constraint, we study a class of fluid-budget balancing policies that aim to balance between remaining close to actions selected by a fluid relaxation optimal policy and remaining close to the shared resource budget. We show that this policy achieves performance within O(1) of optimal under particular nondegeneracy assumptions. This result generalizes recent advances for restless bandit problems by considering (a) any finite number of actions for each subproblem and (b) heterogeneous subproblems with a fixed number of types. We demonstrate the use of these techniques on dynamic multi-warehouse inventory problems and find empirically that these fluid-based policies achieve excellent performance, as our theory suggests.

Keywords: weakly coupled stochastic dynamic programs, fluid relaxations, restless bandits, asymptotic optimality

JEL Classification: C61