Learning to Automatically Spectate Games for Esports Using Object Detection Mechanism

Abstract

Human game observers, who control in-game cameras and provide viewers with an engaging experience, are a vital part of Electronic sports (Esports) which has emerged as a rapidly growing industry in recent years. However, such a professional human observer poses several problems. For example, they are prone to missing events occurring concurrently across the map. Furthermore, human observers are hard to afford when only a small number of spectators are spectating the game. As a result, the need for automatic observers has grown in recent years. To create automatic observers, various rule-based and learning-based methods are explored. So far, these methods are both based on defining in-game events. However, such event-based methods necessitate detailed predefined events, demanding high domain knowledge when developing. Furthermore, they cannot observe events not specifically defined by humans, making them unsuitable for complex games with numerous event cases. In this paper, we propose the method to overcome these problems by utilizing multiple human observational data and an object detection method, Mask R-CNN, in the real-time strategy game (e.g., StarCraft). Our method can observe events that were not predefined by learning from human observing data rather than defining and focusing on specific events as previous automatic observers did. As a result, we show that our automatic observer outperforms both current rule-based methods and human observers. The game observation video that compares our method and the rule-based method is available at https://www.youtube.com/watch?v=61JIfSrLHVk .

Keywords: Esports, Spectators, Automatic Observer, Mask R-CNN, StarCraft