IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v297y2021ics0306261921005511.html
   My bibliography  Save this article

Real-Time interaction of active distribution network and virtual microgrids: Market paradigm and data-driven stakeholder behavior analysis

Author

Listed:
  • Zhu, Ziqing
  • Wing Chan, Ka
  • Bu, Siqi
  • Zhou, Bin
  • Xia, Shiwei

Abstract

In order to incorporate the independent Virtual Microgrids (VMGs) to the real-time operation of upstream active distribution network (ADN), an interactive dispatching model of VMGs and ADN is proposed, in which the downstream VMGs perform self-dispatching while trading both energy and ancillary service procurement to the Distribution System Operator (DSO). The bi-level bidding and market clearing model is modelled as a data-driven Multi-Agent Reinforcement Learning (MARL) with the solution of Win-or-Learn-Fast Policy Hill-Climbing (WoLF-PHC) algorithm, which is an online and fully-distributed training, enabling VMGs to dynamically update their bidding strategies based on previous market clearing results. VMGs would thereafter conduct the economic dispatching considering the conditional value-at-risk (CVaR) of penalties caused by the curtailment of renewables, load loss, and failure of providing energy or ancillary service to DSO. Finally, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to analyze the inherent dynamics of the proposed MARL driven by WoLF-PHC, revealing the relation between VMGs’ bidding strategy convergence and the trading paradigm. The case study validates the advancement of computational performance of WoLF-PHC compared with conventional Q-learning in the aspects of convergence and computation speed, and the impact of risk coefficient on the VMGs’ real-time dispatching strategies is also demonstrated.

Suggested Citation

  • Zhu, Ziqing & Wing Chan, Ka & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2021. "Real-Time interaction of active distribution network and virtual microgrids: Market paradigm and data-driven stakeholder behavior analysis," Applied Energy, Elsevier, vol. 297(C).
    • Handle: RePEc:eee:appene:v:297:y:2021:i:c:s0306261921005511
    DOI: 10.1016/j.apenergy.2021.117107
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261921005511
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2021.117107?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. RUIZ, Carlos & CONEJO, Antonio J. & SMEERS, Yves, 2012. "Equilibria in an oligopolistic electricity pool with stepwise offer curves," LIDAM Reprints CORE 2395, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. Liu, Yangyang & Shen, Zhongqi & Tang, Xiaowei & Lian, Hongbo & Li, Jiarui & Gong, Jinxia, 2019. "Worst-case conditional value-at-risk based bidding strategy for wind-hydro hybrid systems under probability distribution uncertainties," Applied Energy, Elsevier, vol. 256(C).
    3. Jalali, Mehdi & Zare, Kazem & Seyedi, Heresh, 2017. "Strategic decision-making of distribution network operator with multi-microgrids considering demand response program," Energy, Elsevier, vol. 141(C), pages 1059-1071.
    4. Xie, Min & Ji, Xiang & Hu, Xintong & Cheng, Peijun & Du, Yuxin & Liu, Mingbo, 2018. "Autonomous optimized economic dispatch of active distribution system with multi-microgrids," Energy, Elsevier, vol. 153(C), pages 479-489.
    5. Kumar, Abhishek & Meena, Nand K. & Singh, Arvind R. & Deng, Yan & He, Xiangning & Bansal, R.C. & Kumar, Praveen, 2019. "Strategic integration of battery energy storage systems with the provision of distributed ancillary services in active distribution systems," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    6. Hu, Qian & Zhu, Ziqing & Bu, Siqi & Wing Chan, Ka & Li, Fangxing, 2021. "A multi-market nanogrid P2P energy and ancillary service trading paradigm: Mechanisms and implementations," Applied Energy, Elsevier, vol. 293(C).
    7. Wen, Lulu & Zhou, Kaile & Yang, Shanlin & Lu, Xinhui, 2019. "Optimal load dispatch of community microgrid with deep learning based solar power and load forecasting," Energy, Elsevier, vol. 171(C), pages 1053-1065.
    8. Nadeem Javaid & Sakeena Javaid & Wadood Abdul & Imran Ahmed & Ahmad Almogren & Atif Alamri & Iftikhar Azim Niaz, 2017. "A Hybrid Genetic Wind Driven Heuristic Optimization Algorithm for Demand Side Management in Smart Grid," Energies, MDPI, vol. 10(3), pages 1-27, March.
    9. Anvari-Moghaddam, Amjad & Rahimi-Kian, Ashkan & Mirian, Maryam S. & Guerrero, Josep M., 2017. "A multi-agent based energy management solution for integrated buildings and microgrid system," Applied Energy, Elsevier, vol. 203(C), pages 41-56.
    10. Gao, Xiang & Chan, Ka Wing & Xia, Shiwei & Zhou, Bin & Lu, Xi & Xu, Da, 2019. "Risk-constrained offering strategy for a hybrid power plant consisting of wind power producer and electric vehicle aggregator," Energy, Elsevier, vol. 177(C), pages 183-191.
    11. Lv, Tianguang & Ai, Qian, 2016. "Interactive energy management of networked microgrids-based active distribution system considering large-scale integration of renewable energy resources," Applied Energy, Elsevier, vol. 163(C), pages 408-422.
    12. Du, Yan & Wang, Zhiwei & Liu, Guangyi & Chen, Xi & Yuan, Haoyu & Wei, Yanli & Li, Fangxing, 2018. "A cooperative game approach for coordinating multi-microgrid operation within distribution systems," Applied Energy, Elsevier, vol. 222(C), pages 383-395.
    13. Khan, Muhammad Waseem & Wang, Jie, 2017. "The research on multi-agent system for microgrid control and optimization," Renewable and Sustainable Energy Reviews, Elsevier, vol. 80(C), pages 1399-1411.
    14. Tang, Chong & Liu, Mingbo & Xie, Min & Dong, Ping & Zhu, Jianquan & Lin, Shunjiang, 2021. "A single-leader and multiple-follower stackelberg model for the look-ahead dispatch of plug-in electric buses in multiple microgrids," Energy, Elsevier, vol. 214(C).
    15. Wang, Luhao & Zhang, Bingying & Li, Qiqiang & Song, Wen & Li, Guanguan, 2019. "Robust distributed optimization for energy dispatch of multi-stakeholder multiple microgrids under uncertainty," Applied Energy, Elsevier, vol. 255(C).
    16. Elsakaan, Asmaa A. & El-Sehiemy, Ragab A. & Kaddah, Sahar S. & Elsaid, Mohammed I., 2018. "An enhanced moth-flame optimizer for solving non-smooth economic dispatch problems with emissions," Energy, Elsevier, vol. 157(C), pages 1063-1078.
    17. Moradi, Mohammad H. & Razini, Saleh & Mahdi Hosseinian, S., 2016. "State of art of multiagent systems in power engineering: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 58(C), pages 814-824.
    18. Liu, Yixin & Guo, Li & Wang, Chengshan, 2018. "A robust operation-based scheduling optimization for smart distribution networks with multi-microgrids," Applied Energy, Elsevier, vol. 228(C), pages 130-140.
    19. Ajoulabadi, Ata & Ravadanegh, Sajad Najafi & Behnam Mohammadi-Ivatloo,, 2020. "Flexible scheduling of reconfigurable microgrid-based distribution networks considering demand response program," Energy, Elsevier, vol. 196(C).
    20. Grover-Silva, Etta & Heleno, Miguel & Mashayekh, Salman & Cardoso, Gonçalo & Girard, Robin & Kariniotakis, George, 2018. "A stochastic optimal power flow for scheduling flexible resources in microgrids operation," Applied Energy, Elsevier, vol. 229(C), pages 201-208.
    21. Jingpeng Yue & Zhijian Hu & Amjad Anvari-Moghaddam & Josep M. Guerrero, 2019. "A Multi-Market-Driven Approach to Energy Scheduling of Smart Microgrids in Distribution Networks," Sustainability, MDPI, vol. 11(2), pages 1-16, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lei, Zhenxing & Liu, Mingbo & Shen, Zhijun & Lu, Wentian & Lu, Zhilin, 2023. "A data-driven Stackelberg game approach applied to analysis of strategic bidding for distributed energy resource aggregator in electricity markets," Renewable Energy, Elsevier, vol. 215(C).
    2. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    3. Pearson, Simon & Wellnitz, Sonja & Crespo del Granado, Pedro & Hashemipour, Naser, 2022. "The value of TSO-DSO coordination in re-dispatch with flexible decentralized energy sources: Insights for Germany in 2030," Applied Energy, Elsevier, vol. 326(C).
    4. Jun Dong & Yaoyu Zhang & Yuanyuan Wang & Yao Liu, 2021. "A Two-Stage Optimal Dispatching Model for Micro Energy Grid Considering the Dual Goals of Economy and Environmental Protection under CVaR," Sustainability, MDPI, vol. 13(18), pages 1-28, September.
    5. Yanbin Li & Yanting Sun & Junjie Zhang & Feng Zhang, 2022. "Optimal Microgrid System Operating Strategy Considering Variable Wind Power Outputs and the Cooperative Game among Subsystem Operators," Energies, MDPI, vol. 15(18), pages 1-20, September.
    6. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    7. Wang, Jun & Xu, Jian & Ke, Deping & Liao, Siyang & Sun, Yuanzhang & Wang, Jingjing & Yao, Liangzhong & Mao, Beiling & Wei, Congying, 2023. "A tri-level framework for distribution-level market clearing considering strategic participation of electrical vehicles and interactions with wholesale market," Applied Energy, Elsevier, vol. 329(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Diptish Saha & Najmeh Bazmohammadi & Juan C. Vasquez & Josep M. Guerrero, 2023. "Multiple Microgrids: A Review of Architectures and Operation and Control Strategies," Energies, MDPI, vol. 16(2), pages 1-32, January.
    2. Qiu, Haifeng & You, Fengqi, 2020. "Decentralized-distributed robust electric power scheduling for multi-microgrid systems," Applied Energy, Elsevier, vol. 269(C).
    3. Han, Dongho & Lee, Jay H., 2021. "Two-stage stochastic programming formulation for optimal design and operation of multi-microgrid system using data-based modeling of renewable energy sources," Applied Energy, Elsevier, vol. 291(C).
    4. Tang, Chong & Liu, Mingbo & Dai, Yue & Wang, Zhijun & Xie, Min, 2019. "Decentralized saddle-point dynamics solution for optimal power flow of distribution systems with multi-microgrids," Applied Energy, Elsevier, vol. 252(C), pages 1-1.
    5. Kong, Xiangyu & Liu, Dehong & Wang, Chengshan & Sun, Fangyuan & Li, Shupeng, 2020. "Optimal operation strategy for interconnected microgrids in market environment considering uncertainty," Applied Energy, Elsevier, vol. 275(C).
    6. Ju, Liwei & Zhang, Qi & Tan, Zhongfu & Wang, Wei & Xin, He & Zhang, Zehao, 2018. "Multi-agent-system-based coupling control optimization model for micro-grid group intelligent scheduling considering autonomy-cooperative operation strategy," Energy, Elsevier, vol. 157(C), pages 1035-1052.
    7. Nawaz, Arshad & Zhou, Min & Wu, Jing & Long, Chengnian, 2022. "A comprehensive review on energy management, demand response, and coordination schemes utilization in multi-microgrids network," Applied Energy, Elsevier, vol. 323(C).
    8. Li, Longxi, 2021. "Coordination between smart distribution networks and multi-microgrids considering demand side management: A trilevel framework," Omega, Elsevier, vol. 102(C).
    9. Vahabi, Ali Reza & Latify, Mohammad Amin & Rahimiyan, Morteza & Yousefi, G. Reza, 2018. "An equitable and efficient energy management approach for a cluster of interconnected price responsive demands," Applied Energy, Elsevier, vol. 219(C), pages 276-289.
    10. Li, Qiang & Gao, Mengkai & Lin, Houfei & Chen, Ziyu & Chen, Minyou, 2019. "MAS-based distributed control method for multi-microgrids with high-penetration renewable energy," Energy, Elsevier, vol. 171(C), pages 284-295.
    11. Jicheng Liu & Fangqiu Xu & Shuaishuai Lin & Hua Cai & Suli Yan, 2018. "A Multi-Agent-Based Optimization Model for Microgrid Operation Using Dynamic Guiding Chaotic Search Particle Swarm Optimization," Energies, MDPI, vol. 11(12), pages 1-22, November.
    12. Karimi, Hamid & Jadid, Shahram, 2020. "Optimal energy management for multi-microgrid considering demand response programs: A stochastic multi-objective framework," Energy, Elsevier, vol. 195(C).
    13. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    14. Mehdizadeh, Ali & Taghizadegan, Navid & Salehi, Javad, 2018. "Risk-based energy management of renewable-based microgrid using information gap decision theory in the presence of peak load management," Applied Energy, Elsevier, vol. 211(C), pages 617-630.
    15. Phani Raghav, L. & Seshu Kumar, R. & Koteswara Raju, D. & Singh, Arvind R., 2022. "Analytic Hierarchy Process (AHP) – Swarm intelligence based flexible demand response management of grid-connected microgrid," Applied Energy, Elsevier, vol. 306(PB).
    16. Mohseni, Shayan & Pishvaee, Mir Saman, 2023. "Energy trading and scheduling in networked microgrids using fuzzy bargaining game theory and distributionally robust optimization," Applied Energy, Elsevier, vol. 350(C).
    17. Tostado-Véliz, Marcos & Hasanien, Hany M. & Jordehi, Ahmad Rezaee & Turky, Rania A. & Jurado, Francisco, 2023. "Risk-averse optimal participation of a DR-intensive microgrid in competitive clusters considering response fatigue," Applied Energy, Elsevier, vol. 339(C).
    18. Matamala, Yolanda & Feijoo, Felipe, 2021. "A two-stage stochastic Stackelberg model for microgrid operation with chance constraints for renewable energy generation uncertainty," Applied Energy, Elsevier, vol. 303(C).
    19. Hlalele, Thabo G. & Naidoo, Raj M. & Bansal, Ramesh C. & Zhang, Jiangfeng, 2020. "Multi-objective stochastic economic dispatch with maximal renewable penetration under renewable obligation," Applied Energy, Elsevier, vol. 270(C).
    20. Giulia Maesano & Giuseppe Di Vita & Gaetano Chinnici & Gioacchino Pappalardo & Mario D'Amico, 2020. "The Role of Credence Attributes in Consumer Choices of Sustainable Fish Products: A Review," Sustainability, MDPI, vol. 12(23), pages 1-18, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:297:y:2021:i:c:s0306261921005511. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.