Multi-agent deep reinforcement learning-based cooperative energy management for regional integrated energy system incorporating active demand-side management

Incorporating multiple flexible resources and energy sharing into the regional integrated energy system (RIES) provides an attractive pathway for resilience enhancement. However, traditional model-based optimization methods are not sufficiently flexible to deal with benefit games of multiple entities and complex multi-energy flows of RIES. Therefore, this work proposes a cooperative energy management framework using multi-agent deep reinforcement learning (MADRL) for optimal operation. Firstly, the collaborative optimization between shared energy storage, IES energy stations and users is developed, in which users could make subjective decisions to participate in demand response and the shared energy storage is employed to coordinate energy balance. Secondly, the cooperative optimization is formulated as a Markov decision process. The multi-agent twin delayed deep deterministic policy (MATD3) is leveraged to tackle the optimal scheduling problem, aiming at operation profits and user satisfaction. Thirdly, an imitation actor-attention critic (IAAC) mechanism is proposed, which could assist actors in learning effective strategies and generate more accurate state-action value function of critics. The results show that the proposed IAAC-MATD3 algorithm exhibits the fastest convergence compared with baseline algorithms. The operation cost of cooperation optimization is better than those of three baseline scenarios and achieves an improvement of 43.7 %, 19.9 %, and 34.6 %, respectively.