Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates

This paper presents a novel data-driven approach that leverages reinforcement learning to enhance the efficiency and safety of existing energy flexibility controllers, addressing challenges posed by the dynamic and uncertain nature of modern energy landscapes. With the increasing integration of renewable energy sources, conventional controllers struggle to maintain both safety and optimality. Our proposed approach introduces two significant contributions to standard RL approaches: a data-driven predictive safety filter and an online changepoint detection and policy updating module. Through continuous constraint satisfaction, the predictive safety filter guarantees absolute safety of the proposed controller. Meanwhile, the changepoint detection and policy updating module, inspired by the concept of continual learning, enhances the controller’s adaptivity to non-stationary environments. By identifying changes in the environment, it triggers relearning of the agent, making the controller resilient to evolving conditions. Validation of our approach is conducted on a grid-connected PV-battery-load system, demonstrating its effectiveness in simultaneously improving safety and performance over traditional learning methods. More specifically, the proposed solution was able to increase the energy flexibility by reducing energy costs with 9.3%.