Suppressing active power fluctuations at PCC in grid-connection microgrids via multiple BESSs: A collaborative multi-agent reinforcement learning approach

In recent years, with the increasing proportion of photovoltaic (PV) power generation in grid-connected microgrids, suppressing power fluctuations at the point of common coupling (PCC) has become a challenge. This paper proposes a collaborative power dispatch algorithm for battery energy storage systems (BESSs) based on multi-agent reinforcement learning (MARL), aiming to suppress the PCC power fluctuations caused by the uncertainty of PV power generation. First, a distributed multi-agent communication framework is developed, which defines the neighboring areas of agents based on the physical distances between BESSs to reduce the communication and computational cost of agents. Subsequently, a distributed multi-agent dueling double deep Q-network power dispatch algorithm based on the communication framework is proposed. In the proposed algorithm, a distributed Markov decision process is designed, enabling agents to share actions and rewards with neighboring agents locally to collaboratively learn optimal charging and discharging actions for suppress PCC power fluctuations. Finally, the scalability and effectiveness of the proposed algorithm in suppressing PCC power and voltage fluctuations and reducing operational cost are validated through simulation experiments based on the IEEE-33 bus and IEEE-141 bus systems. The simulation results demonstrate significant advantages of the proposed algorithm compared with other baseline MARL and traditional optimization methods.