Data-Driven Knowledge Transfer in Batch Q* Learning

48 Pages Posted: 16 Apr 2024

See all articles by Elynn Chen

Elynn Chen

New York University (NYU) - Leonard N. Stern School of Business

Xi Chen

New York University (NYU) - Leonard N. Stern School of Business

Wenbo Jing

New York University (NYU) - Leonard N. Stern School of Business

Date Written: April 2, 2024

Abstract

In data-driven decision-making in marketing, healthcare, and education, it is desirable to utilize a large amount of data from existing ventures to navigate high-dimensional feature spaces and address data scarcity in new ventures. We explore knowledge transfer in dynamic decision-making by concentrating on batch stationary environments and formally defining task discrepancies through the lens of Markov decision processes (MDPs). We propose a framework of Transferred Fitted $Q$-Iteration algorithm with general function approximation, enabling the direct estimation of the optimal action-state function $Q^*$ using both target and source data. We establish the relationship between statistical performance and MDP task discrepancy under sieve approximation, shedding light on the impact of source and target sample sizes and task discrepancy on the effectiveness of knowledge transfer. We show that the final learning error of the $Q^*$ function is significantly improved from the single task rate both theoretically and empirically.

Keywords: Transfer Learning; Stationary Markov Decision Process; Offline Reinforcement Learning; Sieve Approximation; Fitted $Q$ Iteration

Suggested Citation

Chen, Elynn and Chen, Xi and Jing, Wenbo, Data-Driven Knowledge Transfer in Batch Q* Learning (April 2, 2024). Available at SSRN: https://ssrn.com/abstract=4787816 or http://dx.doi.org/10.2139/ssrn.4787816

Elynn Chen (Contact Author)

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

Xi Chen

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

Wenbo Jing

New York University (NYU) - Leonard N. Stern School of Business ( email )

44 West 4th Street
Suite 9-160
New York, NY NY 10012
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
83
Abstract Views
265
Rank
613,816
PlumX Metrics