Reinforcement Learning Swarm of Self-Organizing Unmanned Surface Vehicles with Unavailable Dynamics

Abstract

In this paper, self-organizing a swarm of unmanned surface vehicles (USVs) is creatively solved by optimal reinforcement learning control approach without acquiring USV dynamics. Key contributions are unfolded as follows: (1) Self-organizing swarm mechanism for flexible decision-making is established by combing control theory with repulsive potentials; (2) Distributed finite-time extended state observers are devised to sufficiently accommodate multiple unknowns covering internal dynamics and external uncertainties/disturbances; and (3) Swarming coordination of individual USV is optimally synthesized by reinforcement learning-based distributed control architecture. As a consequence, semi-global asymptotic stability of the entire self-organizing swarm can be rigorously ensured by Lyapunov analysis. Virtual-reality experiments show that reinforcement learning swarm is skilled in coordinating a herd of USVs tracking a trajectory.

Keywords: Unmanned surface vehicle swarm, Self-organizing mechanism, Reinforcement learning swarm control, Model-free optimal control