Incremental learning user profile and deep reinforcement learning for managing building energy in heating water

Deep reinforcement learning (DRL) has garnered growing attention as a data-driven control technique in the field of built environments. However, the existing DRL approaches for managing water systems cannot consider information from multiple time steps, are prone to overestimation, fall into the problem of locally optimal solutions, and fail to cope with time-varying environments, resulting in an inability to minimize energy consumption while considering water comfort and hygiene of occupants. Therefore, this study proposes an incremental learning user profile and deep reinforcement learning (ILUPDRL) method for controlling hot water systems. This study employs hot water user profiles to reflect the hot water demand (HWD) habits. The proposed ILUPDRL addresses the challenges arising from evolving HWD through incremental learning of hot water user profiles. Moreover, to enable the ILUPDRL to consider information from multiple time steps, this study proposes the recurrent proximal policy optimization (RPPO) algorithm and integrates the RPPO into the ILUPDRL. The simulation results show that the ILUPDRL achieves up to 67.53 % energy savings while considering the water comfort and water hygiene of occupants.