A Novel Safe Multi-Agent Deep Reinforcement Learning-Based Method for Smart Building Energy Management

Abstract

Compared with traditional building power supply strategy, smart building (SB) energy management based on deep reinforcement learning (DRL) method significantly improves the local consumption of renewable energy and enhances economic grid interaction, showing broad application potential. However, the DRL algorithm may produce unsafe agent actions, which severely affect the stable power supply of SB. Therefore, this paper proposes a safe multi-agent DRL-based method for SB energy management. In the proposed method, SB is decomposed into several energy-local area networks (E-LANs), thereby SB energy management can be achieved by the coordinated energy management of all E-LANs. The safety of action is ensured by safety filter (SF), while the extent of action violating SF boundary is incorporated into the reward function to guide the behavior of agents. Subsequently, the safety evaluation of action is input to the critic network, enabling the algorithm to learn the pattern of taking safe actions. Simulation results show that, compared with original multi-agent deep deterministic policy gradient (MADDPG) and MADDPG with SF only, the proposed method reduces the total cost of SB energy management in a scheduling period by 15.3% and 19.5%, respectively, while maintaining the final state-of-charges of energy storage units near the target values.

Keywords: Multi-agent deep reinforcement learning, Safe deep reinforcement learning, Smart building, Energy Management