A Tour of Reinforcement Learning: The View from Continuous Control

Posted: 24 May 2019

See all articles by Benjamin Recht

Benjamin Recht

University of California, Berkeley - Department of Electrical Engineering & Computer Sciences (EECS)

Date Written: May 2019

Abstract

This article surveys reinforcement learning from the perspective of optimization and control, with a focus on continuous control applications. It reviews the general formulation, terminology, and typical experimental implementations of reinforcement learning as well as competing solution paradigms. In order to compare the relative merits of various techniques, it presents a case study of the linear quadratic regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. It also describes how merging techniques from learning theory and control can provide nonasymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. The article concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.

Suggested Citation

Recht, Benjamin, A Tour of Reinforcement Learning: The View from Continuous Control (May 2019). Annual Review of Control, Robotics, and Autonomous Systems, Vol. 2, pp. 253-279, 2019, Available at SSRN: https://ssrn.com/abstract=3393370 or http://dx.doi.org/10.1146/annurev-control-053018-023825

Benjamin Recht (Contact Author)

University of California, Berkeley - Department of Electrical Engineering & Computer Sciences (EECS) ( email )

Berkeley, CA 94720-1712
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
221
PlumX Metrics