IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v378y2025ipas0306261924020245.html
   My bibliography  Save this article

Cooperative optimal dispatch of multi-microgrids for low carbon economy based on personalized federated reinforcement learning

Author

Listed:
  • Yang, Ting
  • Xu, Zheming
  • Ji, Shijie
  • Liu, Guoliang
  • Li, Xinhong
  • Kong, Haibo

Abstract

The cooperative optimization dispatch of interconnected multi-microgrid (MMG) systems present broad prospects and significant opportunities for the efficient utilization of large-scale renewable energy resources. These systems facilitate the optimal allocation of energy resources and enhance economic efficiency in operational costs. Nevertheless, divergent interests among heterogeneous microgrid (MG) entities during the cooperative optimization dispatch process lead to obstacles in data sharing and issues with privacy breaches. Additionally, the process is complicated by multi-energy coupling relationships and high-dimensional decision-making, which can result in difficulties achieving convergence and a loss of accuracy in energy management. Furthermore, the lack of operational data and dispatch experience in newly established MGs hinders the ability to rapidly “cold start” dispatch tasks. To fill the above knowledge gap, a cooperative optimization dispatch method for MMG is proposed, which based on personalized federated multi-agent reinforcement learning with clustering (C-PFMARL). This method formulates an optimal low-carbon economic dispatch strategy that incorporates electricity and carbon allowance trading within multiple MG systems. Initially, a cooperative training framework for MMG is constructed under the privacy protection of federated reinforcement learning. This framework allows MMG to train optimization dispatch models based on heterogeneous multi-agent twin delayed deep deterministic policy gradient (HMATD3). With the federated aggregation of model gradient parameters instead of transferring private data, this approach achieves a privacy protection effect of “data cooperation without leaving locality “. Secondly, a dual-ended dynamic clustering algorithm for sharing knowledge within groups is proposed, characterized by model intermediate gradient parameters. It employs a personalized federated transfer strategy based on neural network layering, which enhances the convergence speed and dispatch precision under optimal strategies of the local optimization dispatch model. Moreover, a “cold start” transfer strategy aimed at newly established MG entities is formulated, achieving precise assistance and rapid cold start in optimization dispatch experience. Finally, our case analysis validates the effectiveness and training convergence of the constructed dispatch model. The overall integrated cost of the MMG system has been reduced by 5.78 %, and carbon emissions have decreased by 8.43 %. The dispatch cold-start speed for newly established MGs has improved by 42.83 %, with the optimization results also demonstrating robust economic and low-carbon benefits.

Suggested Citation

  • Yang, Ting & Xu, Zheming & Ji, Shijie & Liu, Guoliang & Li, Xinhong & Kong, Haibo, 2025. "Cooperative optimal dispatch of multi-microgrids for low carbon economy based on personalized federated reinforcement learning," Applied Energy, Elsevier, vol. 378(PA).
  • Handle: RePEc:eee:appene:v:378:y:2025:i:pa:s0306261924020245
    DOI: 10.1016/j.apenergy.2024.124641
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924020245
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2024.124641?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Qiu, Dawei & Xue, Juxing & Zhang, Tingqi & Wang, Jianhong & Sun, Mingyang, 2023. "Federated reinforcement learning for smart building joint peer-to-peer energy and carbon allowance trading," Applied Energy, Elsevier, vol. 333(C).
    2. Rosero, D.G. & Sanabria, E. & Díaz, N.L. & Trujillo, C.L. & Luna, A. & Andrade, F., 2023. "Full-deployed energy management system tested in a microgrid cluster," Applied Energy, Elsevier, vol. 334(C).
    3. Nawaz, Arshad & Zhou, Min & Wu, Jing & Long, Chengnian, 2022. "A comprehensive review on energy management, demand response, and coordination schemes utilization in multi-microgrids network," Applied Energy, Elsevier, vol. 323(C).
    4. Xia, Qinqin & Wang, Yu & Zou, Yao & Yan, Ziming & Zhou, Niancheng & Chi, Yuan & Wang, Qianggang, 2024. "Regional-privacy-preserving operation of networked microgrids: Edge-cloud cooperative learning with differentiated policies," Applied Energy, Elsevier, vol. 370(C).
    5. Zhong, Xiaoqing & Zhong, Weifeng & Liu, Yi & Yang, Chao & Xie, Shengli, 2022. "Optimal energy management for multi-energy multi-microgrid networks considering carbon emission limitations," Energy, Elsevier, vol. 246(C).
    6. Samende, Cephas & Cao, Jun & Fan, Zhong, 2022. "Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints," Applied Energy, Elsevier, vol. 317(C).
    7. Chang, Weiguang & Dong, Wei & Yang, Qiang, 2023. "Day-ahead bidding strategy of cloud energy storage serving multiple heterogeneous microgrids in the electricity market," Applied Energy, Elsevier, vol. 336(C).
    8. Li, Sichen & Hu, Weihao & Cao, Di & Chen, Zhe & Huang, Qi & Blaabjerg, Frede & Liao, Kaiji, 2023. "Physics-model-free heat-electricity energy management of multiple microgrids based on surrogate model-enabled multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 346(C).
    9. Zhao, Xiaohu & Huang, Guohe & Lu, Chen & Zhou, Xiong & Li, Yongping, 2020. "Impacts of climate change on photovoltaic energy potential: A case study of China," Applied Energy, Elsevier, vol. 280(C).
    10. Liao, Wei & Xiao, Fu & Li, Yanxue & Peng, Jinqing, 2024. "Comparative study on electricity transactions between multi-microgrid: A hybrid game theory-based peer-to-peer trading in heterogeneous building communities considering electric vehicles," Applied Energy, Elsevier, vol. 367(C).
    11. Seleshi G. Yalew & Michelle T. H. van Vliet & David E. H. J. Gernaat & Fulco Ludwig & Ariel Miara & Chan Park & Edward Byers & Enrica De Cian & Franziska Piontek & Gokul Iyer & Ioanna Mouratiadou & Ja, 2020. "Impacts of climate change on energy systems in global and regional scenarios," Nature Energy, Nature, vol. 5(10), pages 794-802, October.
    12. Luqin Fan & Jing Zhang & Yu He & Ying Liu & Tao Hu & Heng Zhang, 2021. "Optimal Scheduling of Microgrid Based on Deep Deterministic Policy Gradient and Transfer Learning," Energies, MDPI, vol. 14(3), pages 1-15, January.
    13. Neto, Pedro Bezerra Leite & Saavedra, Osvaldo R. & Oliveira, Denisson Q., 2020. "The effect of complementarity between solar, wind and tidal energy in isolated hybrid microgrids," Renewable Energy, Elsevier, vol. 147(P1), pages 339-355.
    14. Guo, Chenyu & Wang, Xin & Zheng, Yihui & Zhang, Feng, 2022. "Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    15. Mei, H. & Li, Y.P. & Suo, C. & Ma, Y. & Lv, J., 2020. "Analyzing the impact of climate change on energy-economy-carbon nexus system in China," Applied Energy, Elsevier, vol. 262(C).
    16. Han, Dongho & Lee, Jay H., 2021. "Two-stage stochastic programming formulation for optimal design and operation of multi-microgrid system using data-based modeling of renewable energy sources," Applied Energy, Elsevier, vol. 291(C).
    17. Qiu, Haifeng & You, Fengqi, 2020. "Decentralized-distributed robust electric power scheduling for multi-microgrid systems," Applied Energy, Elsevier, vol. 269(C).
    18. Chen, Qi & Kuang, Zhonghong & Liu, Xiaohua & Zhang, Tao, 2024. "Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics," Applied Energy, Elsevier, vol. 364(C).
    19. Du, Yan & Wang, Zhiwei & Liu, Guangyi & Chen, Xi & Yuan, Haoyu & Wei, Yanli & Li, Fangxing, 2018. "A cooperative game approach for coordinating multi-microgrid operation within distribution systems," Applied Energy, Elsevier, vol. 222(C), pages 383-395.
    20. Restrepo, Mauricio & Cañizares, Claudio A. & Simpson-Porco, John W. & Su, Peter & Taruc, John, 2021. "Optimization- and Rule-based Energy Management Systems at the Canadian Renewable Energy Laboratory microgrid facility," Applied Energy, Elsevier, vol. 290(C).
    21. Hu, Jinxing & Li, Hongru, 2022. "A transfer learning-based scenario generation method for stochastic optimal scheduling of microgrid with newly-built wind farm," Renewable Energy, Elsevier, vol. 185(C), pages 1139-1151.
    22. Nawaz, Arshad & Wu, Jing & Ye, Jun & Dong, Yidi & Long, Chengnian, 2023. "Distributed MPC-based energy scheduling for islanded multi-microgrid considering battery degradation and cyclic life deterioration," Applied Energy, Elsevier, vol. 329(C).
    23. Perera, A.T.D. & Wang, Z. & Nik, Vahid M. & Scartezzini, Jean-Louis, 2021. "Towards realization of an Energy Internet: Designing distributed energy systems using game-theoretic approach," Applied Energy, Elsevier, vol. 283(C).
    24. Abid, Md. Shadman & Apon, Hasan Jamil & Hossain, Salman & Ahmed, Ashik & Ahshan, Razzaqul & Lipu, M.S. Hossain, 2024. "A novel multi-objective optimization based multi-agent deep reinforcement learning approach for microgrid resources planning," Applied Energy, Elsevier, vol. 353(PA).
    25. Kang, Dongju & Kang, Doeun & Hwangbo, Sumin & Niaz, Haider & Lee, Won Bo & Liu, J. Jay & Na, Jonggeol, 2023. "Optimal planning of hybrid energy storage systems using curtailed renewable energy through deep reinforcement learning," Energy, Elsevier, vol. 284(C).
    26. Ren, Kezheng & Liu, Jun & Wu, Zeyang & Liu, Xinglei & Nie, Yongxin & Xu, Haitao, 2024. "A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters," Applied Energy, Elsevier, vol. 355(C).
    27. Zhang, Yue & Wu, Qiong & Ren, Hongbo & Li, Qifen & Zhou, Weisheng, 2024. "Optimal operation of multi-microgrid systems considering multi-level energy-certificate-carbon coupling trading," Renewable Energy, Elsevier, vol. 227(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rodriguez, Mauricio & Arcos-Aviles, Diego & Guinjoan, Francesc, 2024. "Simple fuzzy logic-based energy management for power exchange in isolated multi-microgrid systems: A case study in a remote community in the Amazon region of Ecuador," Applied Energy, Elsevier, vol. 357(C).
    2. Cheng, Qian & Liu, Pan & Xia, Qian & Cheng, Lei & Ming, Bo & Zhang, Wei & Xu, Weifeng & Zheng, Yalian & Han, Dongyang & Xia, Jun, 2023. "An analytical method to evaluate curtailment of hydro–photovoltaic hybrid energy systems and its implication under climate change," Energy, Elsevier, vol. 278(C).
    3. Makasis, Nikolas & Gu, Xiaoying & Kreitmair, Monika J. & Narsilio, Guillermo A. & Choudhary, Ruchi, 2023. "Geothermal pavements: A city-scale investigation on providing sustainable heating for the city of Cardiff, UK," Renewable Energy, Elsevier, vol. 218(C).
    4. Lan, Xinyao & Jin, Jiahui & Xu, Beibei & Chen, Diyi & Egusquiza, Mònica & Kim, Jin-Hyuk & Egusquiza, Eduard & Jafar, Nejadali & Xu, Lin & Kuang, Yuan, 2022. "Physical model test and parametric optimization of a hydroelectric generating system with a coaxial shaft surge tank," Renewable Energy, Elsevier, vol. 200(C), pages 880-899.
    5. Oyewo, Ayobami S. & Aghahosseini, Arman & Movsessian, Maria M. & Breyer, Christian, 2024. "A novel geothermal-PV led energy system analysis on the case of the central American countries Guatemala, Honduras, and Costa Rica," Renewable Energy, Elsevier, vol. 221(C).
    6. Perera, A.T.D. & Khayatian, F. & Eggimann, S. & Orehounig, K. & Halgamuge, Saman, 2022. "Quantifying the climate and human-system-driven uncertainties in energy planning by using GANs," Applied Energy, Elsevier, vol. 328(C).
    7. Abel, Dennis & Lieth, Jonas & Jünger, Stefan, 2024. "Mapping the spatial turn in social science energy research. A computational literature review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 201(C).
    8. Pezalla, Simon & Obringer, Renee, 2023. "Evaluating the household-level climate-electricity nexus across three cities through statistical learning techniques," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    9. Cheng, Qian & Liu, Pan & Xia, Jun & Ming, Bo & Cheng, Lei & Chen, Jie & Xie, Kang & Liu, Zheyuan & Li, Xiao, 2022. "Contribution of complementary operation in adapting to climate change impacts on a large-scale wind–solar–hydro system: A case study in the Yalong River Basin, China," Applied Energy, Elsevier, vol. 325(C).
    10. Barckholtz, Timothy A. & Taylor, Kevin M. & Narayanan, Sundar & Jolly, Stephen & Ghezel-Ayagh, Hossein, 2022. "Molten carbonate fuel cells for simultaneous CO2 capture, power generation, and H2 generation," Applied Energy, Elsevier, vol. 313(C).
    11. Chen, Shiyu & Ma, Chiye & Wang, Wei & Zio, Enrico, 2024. "An agent-based cooperative co-evolutionary framework for optimizing the production planning of energy supply chains under uncertainty scenarios," International Journal of Production Economics, Elsevier, vol. 277(C).
    12. Hosseini Dolatabadi, Sayed Hamid & Bhuiyan, Tanveer Hossain & Chen, Yang & Morales, Jose Luis, 2024. "A stochastic game-theoretic optimization approach for managing local electricity markets with electric vehicles and renewable sources," Applied Energy, Elsevier, vol. 368(C).
    13. Li, Ling-Ling & Ji, Bing-Xiang & Liu, Guan-Chen & Yuan, Jian-Ping & Tseng, Shuan-Wei & Lim, Ming K. & Tseng, Ming-Lang, 2024. "Grid-connected multi-microgrid system operational scheduling optimization: A hierarchical improved marine predators algorithm," Energy, Elsevier, vol. 294(C).
    14. Gebara, C.H. & Laurent, A., 2023. "National SDG-7 performance assessment to support achieving sustainable energy for all within planetary limits," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    15. Kandasamy, Jeevitha & Ramachandran, Rajeswari & Veerasamy, Veerapandiyan & Irudayaraj, Andrew Xavier Raj, 2024. "Distributed leader-follower based adaptive consensus control for networked microgrids," Applied Energy, Elsevier, vol. 353(PA).
    16. Li, Ling-Ling & Ji, Bing-Xiang & Li, Zhong-Tao & Lim, Ming K. & Sethanan, Kanchana & Tseng, Ming-Lang, 2025. "Microgrid energy management system with degradation cost and carbon trading mechanism: A multi-objective artificial hummingbird algorithm," Applied Energy, Elsevier, vol. 378(PA).
    17. Alizadeh, Ali & Kamwa, Innocent & Moeini, Ali & Mohseni-Bonab, Seyed Masoud, 2023. "Energy management in microgrids using transactive energy control concept under high penetration of Renewables; A survey and case study," Renewable and Sustainable Energy Reviews, Elsevier, vol. 176(C).
    18. Fioriti, Davide & Frangioni, Antonio & Poli, Davide, 2021. "Optimal sizing of energy communities with fair revenue sharing and exit clauses: Value, role and business model of aggregators and users," Applied Energy, Elsevier, vol. 299(C).
    19. Zhang, Yiwen & Lin, Rui & Mei, Zhen & Lyu, Minghao & Jiang, Huaiguang & Xue, Ying & Zhang, Jun & Gao, David Wenzhong, 2024. "Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties," Applied Energy, Elsevier, vol. 376(PA).
    20. Zhang, Yue & Wu, Qiong & Ren, Hongbo & Li, Qifen & Zhou, Weisheng, 2024. "Optimal operation of multi-microgrid systems considering multi-level energy-certificate-carbon coupling trading," Renewable Energy, Elsevier, vol. 227(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:378:y:2025:i:pa:s0306261924020245. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.