General Multi-Agent Reinforcement Learning Integrating Heuristic-Based Delay Priority Strategy for Demand and Capacity Balancing

56 Pages Posted: 8 Oct 2022

See all articles by YUTONG CHEN

YUTONG CHEN

Nanjing University of Aeronautics and Astronautics

Yan Xu

Cranfield University

Minghua Hu

Nanjing University of Aeronautics and Astronautics

Abstract

Reinforcement learning (RL) techniques have been studied for solving the demand and capacity balancing (DCB) problem in air traffic management to exploit their full computational potential. Due to the lack of generalisation and the seemingly reduced optimisation performance affected by the training scenarios, it is challenging for existing RL-based DCB methods to be effectively applied in practice. This paper proposes a general multi-agent reinforcement learning (MARL) method that integrates a heuristic-based delay priority strategy to improve the efficiency of the solution and the generalisation of the model. The delay priority strategy is used to reduce the potential learning task and thus training difficulty. This study explores what features of the delay priority strategy are better suited to the MARL method. A long short-term memory (LSTM) network is integrated into a deep q-learning network (DQN) to ensure the model compatible with arbitrary DCB instances and to facilitate agents to identify key sectors. This study is conducted as a part of a large-scale European DCB research project, where real data from French and Spanish airspace are used for experimentation. Results suggest that the proposed method has advantages in generalisation, optimisation performance and computational performance over state-of-the-art RL-based DCB methods.

Keywords: Demand and capacity balancing, Air traffic flow management, Multi-agent reinforcement learning, Heuristic algorithm, Deep q-learning network, Long short-term memory

Suggested Citation

CHEN, YUTONG and Xu, Yan and Hu, Minghua, General Multi-Agent Reinforcement Learning Integrating Heuristic-Based Delay Priority Strategy for Demand and Capacity Balancing. Available at SSRN: https://ssrn.com/abstract=4241524 or http://dx.doi.org/10.2139/ssrn.4241524

YUTONG CHEN

Nanjing University of Aeronautics and Astronautics ( email )

Yudao Street
210016
Nanjing,, 210016
China

Yan Xu (Contact Author)

Cranfield University ( email )

Cranfield
Bedfordshire MK43 OAL, MK43 0AL
United Kingdom

Minghua Hu

Nanjing University of Aeronautics and Astronautics ( email )

Yudao Street
210016
Nanjing,, 210016
China

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
28
Abstract Views
151
PlumX Metrics