Optimal Continuous Control of Refrigerator for Electricity Cost Minimization - Hierarchical Reinforcement Learning Approach

Abstract

A refrigerator is a commonly used household appliance; however, limited research has focused on optimizing temperature control policy with the consideration of Time-of-Use (ToU) electricity price. This paper introduces a novel framework based on hierarchical reinforcement learning (HRL) to control the intensity of refrigerator motors. The objective is to achieve both temperature regulation and cost savings under ToU and stochastic usage patterns. The problem is tackled by two HRL agents, involving the high-level agent responsible for determining temperature reference based on ToU and the low-level agent for adjusting the motor intensity to meet the temperature reference. To tackle non-stationarity in HRL, the high-level agent employs hindsight action transition and reward function approximation, while the low-level agent employs hindsight goal transition. Through the experimental evaluation, the proposed method exhibits superior performance by achieving the lowest total cost, surpassing standard control methods and standard reinforcement learning approaches, resulting in a significantly reduced cost by 5-24%.

Keywords: Hierarchical reinforcement learningRefrigeratorEnergy managementOptimal continuous controlTime-of-Use