Online Planning with Offline Simulation

45 Pages Posted: 27 Nov 2020

See all articles by Wang Chi Cheung

Wang Chi Cheung

National University of Singapore (NUS)

Guodong Lyu

National University of Singapore (NUS) - NUS Business School

Chung-Piaw Teo

NUS Business School - Department of Decision Sciences

Hai Wang

Carnegie Mellon University - Heinz College of Information Systems and Public Policy; Singapore Management University - School of Information Systems

Date Written: October 12, 2020

Abstract

One of the central issues in (finite horizon) online planning problems is to synthesize the impact of real time decisions on the subsequent states of the system, and the performance in the remaining time horizon (cost-to-go function). A complete resolution often leads to intractable dynamic programming problems. In this paper, we propose a computationally efficient approach to this problem that attains near-optimal performance in non-stationary environments. More specifically, we study a general class of online planning problems with concave objective functions and (global) feasibility constraints. A wide range of problems in supply chain management, online advertising, and network revenue management etc., can be appropriately modelled using this online planning framework. Leveraging on the value of the "gradient" information obtained from offline simulation (generated from the distributional information), we develop a generic approach to facilitate online planning for this class of problems. Furthermore, our proposed approach produces near optimal solution with sublinear regret and satisfies the feasibility constraints with high probability. We present extensive numerical evidence to validate the performance of this approach, and discuss its improvement over existing techniques that assume the underlying environment is stationary.

Keywords: Online Planning; Non-Stationary Environment; Distributional Information; Offline Simulation

Suggested Citation

Cheung, Wang Chi and Lyu, Guodong and Teo, Chung-Piaw and Wang, Hai, Online Planning with Offline Simulation (October 12, 2020). Available at SSRN: https://ssrn.com/abstract=3709882 or http://dx.doi.org/10.2139/ssrn.3709882

Wang Chi Cheung

National University of Singapore (NUS) ( email )

1E Kent Ridge Road
NUHS Tower Block Level 7
Singapore, 119228
Singapore

Guodong Lyu (Contact Author)

National University of Singapore (NUS) - NUS Business School ( email )

15 Kent Ridge Drive
Singapore, 119245
Singapore

Chung-Piaw Teo

NUS Business School - Department of Decision Sciences ( email )

15 Kent Ridge Drive
Mochtar Riady Building, BIZ 1 8-69
119245
Singapore

Hai Wang

Carnegie Mellon University - Heinz College of Information Systems and Public Policy ( email )

5000 Forbes Avenue
Pittsburgh, PA 15213-3890
United States

Singapore Management University - School of Information Systems ( email )

School of Information Systems
80 Stamford Road
Singapore 178902, 178899
Singapore

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
99
Abstract Views
405
rank
323,806
PlumX Metrics