Q-Learning and SARSA: A Comparison between Two Intelligent Stochastic Control Approaches for Financial Trading

25 Pages Posted: 17 Jun 2015

See all articles by Marco Corazza

Marco Corazza

Ca Foscari University of Venice - Dipartimento di Economia

Andrea Sangalli

Independent

Date Written: June 10, 2015

Abstract

The purpose of this paper is to solve a stochastic control problem consisting of optimizing the management of a trading system. Two model free machine learning algorithms based on Reinforcement Learning method are compared: the Q-Learning and the SARSA ones. Both these models optimize their behaviours in real time on the basis of the reactions they get from the environment in which operate. This idea is based on a new emerging theory about the market efficiency, the Adaptive Market Hypothesis. We apply the algorithms on single stock price time series using simple state variables. These algorithms operate selecting an action among three possible ones: buy, sell and stay out from the market. We perform several applications based on different parameter settings that are tested on an artificial daily stock prices time series and on different real ones from Italian stock market. Furthermore, performances are both gross and net of transaction costs.

Keywords: Financial trading system, Adaptive Market Hypothesis, model free machine learning, Reinforcement Learning, Q-Learning, SARSA, Italian stock market

JEL Classification: C61, C63, G11

Suggested Citation

Corazza, Marco and Sangalli, Andrea, Q-Learning and SARSA: A Comparison between Two Intelligent Stochastic Control Approaches for Financial Trading (June 10, 2015). University Ca' Foscari of Venice, Dept. of Economics Research Paper Series No. No. 15/WP/2015, Available at SSRN: https://ssrn.com/abstract=2617630 or http://dx.doi.org/10.2139/ssrn.2617630

Marco Corazza (Contact Author)

Ca Foscari University of Venice - Dipartimento di Economia ( email )

Cannaregio 873
Venice, 30121
Italy

Andrea Sangalli

Independent ( email )

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1,328
Abstract Views
4,312
Rank
27,777
PlumX Metrics