Deep reinforcement learning-based joint load scheduling for household multi-energy system

Under the background of the popularization of renewable energy sources and gas-fired domestic devices in households, this paper proposes a joint load scheduling strategy for household multi-energy system (HMES) aiming at minimizing residents’ energy cost while maintaining the thermal comfort. Specifically, the studied HMES contains photovoltaic, gas-electric hybrid heating system, gas-electric kitchen stove and various types of conventional loads. Yet, it is challenging to develop an efficient energy scheduling strategy due to the uncertainties in energy price, photovoltaic generation, outdoor temperature, and residents’ hot water demand. To tackle this problem, we formulate the HMES scheduling problem as a Markov decision process with both continuous and discrete actions and propose a deep reinforcement learning-based HMES scheduling approach. A mixed distribution is used to approximate the scheduling strategies of different types of household devices, and proximal policy optimization is used to optimize the scheduling strategies without requiring any prediction information or distribution knowledge of system uncertainties. The proposed approach can handle continuous actions of power-shiftable devices and discrete actions of time-shiftable devices simultaneously, as well as the optimal management of electrical devices and gas-fired devices, so as to jointly optimize the operation of all household loads. The proposed approach is compared with a deep Q network (DQN)-based approach and a model predictive control (MPC)-based approach. Comparison results show that the average energy cost of the proposed approach is reduced by 12.17% compared to the DQN-based approach and 4.59% compared to the MPC-based approach.