Learning Indirect Actions in Complex Domains: Action Suggestions for Air Traffic Control

Advances in Complex Systems, Vol. 12, Nos. 4-5, pp. 493-512, 2009

Posted: 26 Apr 2010

Date Written: August 1, 2009

Abstract

Providing intelligent algorithms to manage the ever-increasing flow of air traffic is critical to the efficiency and economic viability of air transportation systems. Yet, current automated solutions leave existing human controllers "out of the loop" rendering the potential solutions both technically dangerous (e.g. inability to react to suddenly developing conditions) and politically charged (e.g. role of air traffic controllers in a fully automated system). Instead, this paper outlines a distributed agent-based solution where agents provide suggestions to human controllers. Though conceptually pleasing, this approach introduces two critical research issues. First, the agent actions are now filtered through interactions with other agents, human controllers and the environment before leading to a system state. This indirect action-to-effect process creates a complex learning problem. Second, even in the best case, not all air traffic controllers will be willing or able to follow the agents' suggestions. This partial participation effect will require the system to be robust to the number of controllers that follow the agent suggestions. In this paper, we present an agent reward structure that allows agents to learn good actions in this indirect environment, and explore the ability of those suggestion agents to achieve good system level performance. We present a series of experiments based on real historical air traffic data combined with simulation of air traffic flow around the New York city area. Results show that the agents can improve system-wide performance by up to 20% over that of human controllers alone, and that these results degrade gracefully when the number of human controllers that follow the agents' suggestions declines.

Keywords: Air traffic control, learning indirect actions, suggestion agents, multiagent learning

Suggested Citation

Tumer, Kagan, Learning Indirect Actions in Complex Domains: Action Suggestions for Air Traffic Control (August 1, 2009). Advances in Complex Systems, Vol. 12, Nos. 4-5, pp. 493-512, 2009, Available at SSRN: https://ssrn.com/abstract=1504439

Kagan Tumer (Contact Author)

Oregon State University ( email )

Bexell Hall 200
Corvallis, OR 97331
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Abstract Views
328
PlumX Metrics