Brain-Inspired Theory of Mind Spiking Neural Network Elevates Multi-Agent Cooperation and Competition

Abstract

Theory of mind (ToM), a kind of high social cognitive ability, enables individuals to infer others’ mental states and thus explain and predict others’ behaviors. During dynamic social interaction, inferring and predicting others’ behaviors through ToM is crucial for obtaining more benefits in cooperative and competitive tasks. However, most existing multi-agent decision-making methods select behaviors based on agents’ observations, lacking in-depth inspiration from the ToM mechanism in the brain and thus limiting the performance of multi-agent decision-making. In this paper, we proposed a Multi-Agent Theory of Mind Spiking Neural Network (MAToM-SNN) model, which consists of a ToM model and a decision-making module. We designed two brain-inspired ToM modules (Self-ToM and Other-ToM) to predict others’ behavior based on self-experience and observations of others, respectively. Then, each agent will adjust its behavior according to the predicted actions of others. The proposed model has been verified in multi-agent decision-making environments with cooperative and competitive tasks. The experimental results show that introducing ToM and SNNs could improve cooperation efficiency and help achieve higher rewards compared to multi-agent reinforcement learning models. Especially for competitive tasks, the agents with ToM could improve the rewards of their team by predicting the actions of other agents.

Keywords: Theory of Mind, Multi-Agent Reinforcement Learning, Spiking
 Neural Network, Social Cooperation and Competition