IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2024i20p3241-d1500246.html
   My bibliography  Save this article

Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review

Author

Listed:
  • Lefeng Cheng

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China)

  • Xin Wei

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China
    These authors contributed equally to this work.)

  • Manling Li

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China
    These authors contributed equally to this work.)

  • Can Tan

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China
    These authors contributed equally to this work.)

  • Meng Yin

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China
    These authors contributed equally to this work.)

  • Teng Shen

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China)

  • Tao Zou

    (School of Mechanical and Electrical Engineering, Guangzhou University, Guangzhou 510006, China)

Abstract

With the rapid development of smart grids, the strategic behavior evolution in user-side electricity market transactions has become increasingly complex. To explore the dynamic evolution mechanisms in this area, this paper systematically reviews the application of evolutionary game theory in user-side electricity markets, focusing on its unique advantages in modeling multi-agent interactions and dynamic strategy optimization. While evolutionary game theory excels in explaining the formation of long-term stable strategies, it faces limitations when dealing with real-time dynamic changes and high-dimensional state spaces. Thus, this paper further investigates the integration of deep reinforcement learning, particularly the deep Q -learning network (DQN), with evolutionary game theory, aiming to enhance its adaptability in electricity market applications. The introduction of the DQN enables market participants to perform adaptive strategy optimization in rapidly changing environments, thereby more effectively responding to supply–demand fluctuations in electricity markets. Through simulations based on a multi-agent model, this study reveals the dynamic characteristics of strategy evolution under different market conditions, highlighting the changing interaction patterns among participants in complex market environments. In summary, this comprehensive review not only demonstrates the broad applicability of evolutionary game theory in user-side electricity markets but also extends its potential in real-time decision making through the integration of modern algorithms, providing new theoretical foundations and practical insights for future market optimization and policy formulation.

Suggested Citation

  • Lefeng Cheng & Xin Wei & Manling Li & Can Tan & Meng Yin & Teng Shen & Tao Zou, 2024. "Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review," Mathematics, MDPI, vol. 12(20), pages 1-56, October.
    • Handle: RePEc:gam:jmathe:v:12:y:2024:i:20:p:3241-:d:1500246
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/20/3241/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/20/3241/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Xie, Guangrui & Chen, Xi & Weng, Yang, 2021. "Enhance load forecastability: Optimize data sampling policy by reinforcing user behaviors," European Journal of Operational Research, Elsevier, vol. 295(3), pages 924-934.
    2. Hu, Fanshuai & Zhou, Dequn & Zhu, Qingyuan & Wang, Qunwei, 2024. "How dynamic renewable portfolio standards affect trading behavior of power generators? Considering green certificate and reward/penalty mechanism," Applied Energy, Elsevier, vol. 375(C).
    3. Alex Chamba & Carlos Barrera-Singaña & Hugo Arcos, 2023. "Optimal Reactive Power Dispatch in Electric Transmission Systems Using the Multi-Agent Model with Volt-VAR Control," Energies, MDPI, vol. 16(13), pages 1-25, June.
    4. Lee, Won-Poong & Han, Dongjun & Won, Dongjun, 2022. "Grid-Oriented Coordination Strategy of Prosumers Using Game-theoretic Peer-to-Peer Trading Framework in Energy Community," Applied Energy, Elsevier, vol. 326(C).
    5. Nweye, Kingsley & Sankaranarayanan, Siva & Nagy, Zoltan, 2023. "MERLIN: Multi-agent offline and transfer learning for occupant-centric operation of grid-interactive communities," Applied Energy, Elsevier, vol. 346(C).
    6. Alam, Khandoker Shahjahan & Kaif, A.M.A. Daiyan & Das, Sajal K., 2024. "A blockchain-based optimal peer-to-peer energy trading framework for decentralized energy management with in a virtual power plant: Lab scale studies and large scale proposal," Applied Energy, Elsevier, vol. 365(C).
    7. Tan, Bifei & Lin, Zhenjia & Zheng, Xiaodong & Xiao, Fu & Wu, Qiuwei & Yan, Jinyue, 2023. "Distributionally robust energy management for multi-microgrids with grid-interactive EVs considering the multi-period coupling effect of user behaviors," Applied Energy, Elsevier, vol. 350(C).
    8. Nguyen, Su & Peng, Wei & Sokolowski, Peter & Alahakoon, Damminda & Yu, Xinghuo, 2018. "Optimizing rooftop photovoltaic distributed generation with battery storage for peer-to-peer energy trading," Applied Energy, Elsevier, vol. 228(C), pages 2567-2580.
    9. Li, Xiangyu & Luo, Fengji & Li, Chaojie, 2024. "Multi-agent deep reinforcement learning-based autonomous decision-making framework for community virtual power plants," Applied Energy, Elsevier, vol. 360(C).
    10. Yan, Sizhe & Wang, Weiqing & Li, Xiaozhu & Maimaiti, Pakezhati & Zhao, Yi, 2024. "Cross-regional green certificate transaction strategies based on a double-layer game model," Applied Energy, Elsevier, vol. 356(C).
    11. Jufri, Fauzan Hanif & Widiputra, Victor & Jung, Jaesung, 2019. "State-of-the-art review on power grid resilience to extreme weather events: Definitions, frameworks, quantitative assessment methodologies, and enhancement strategies," Applied Energy, Elsevier, vol. 239(C), pages 1049-1065.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barone, G. & Buonomano, A. & Forzano, C. & Palombo, A. & Russo, G., 2023. "The role of energy communities in electricity grid balancing: A flexible tool for smart grid power distribution optimization," Renewable and Sustainable Energy Reviews, Elsevier, vol. 187(C).
    2. Dimitar Bozalakov & Mohannad J. Mnati & Joannes Laveyne & Jan Desmet & Lieven Vandevelde, 2019. "Battery Storage Integration in Voltage Unbalance and Overvoltage Mitigation Control Strategies and Its Impact on the Power Quality," Energies, MDPI, vol. 12(8), pages 1-26, April.
    3. Yu, Min Gyung & Pavlak, Gregory S., 2023. "Risk-aware sizing and transactive control of building portfolios with thermal energy storage," Applied Energy, Elsevier, vol. 332(C).
    4. Ziad Ragab & Ehsan Pashajavid & Sumedha Rajakaruna, 2024. "Optimal Sizing and Economic Analysis of Community Battery Systems Considering Sensitivity and Uncertainty Factors," Energies, MDPI, vol. 17(18), pages 1-20, September.
    5. Yao He & Yongchun Yang & Meimei Wang & Xudong Zhang, 2022. "Resilience Analysis of Container Port Shipping Network Structure: The Case of China," Sustainability, MDPI, vol. 14(15), pages 1-17, August.
    6. Wu, Chun & Chen, Xingying & Hua, Haochen & Yu, Kun & Gan, Lei & Shen, Jun & Ding, Yi, 2024. "Peer-to-peer energy trading optimization for community prosumers considering carbon cap-and-trade," Applied Energy, Elsevier, vol. 358(C).
    7. Lyu, Cheng & Jia, Youwei & Xu, Zhao, 2021. "Fully decentralized peer-to-peer energy sharing framework for smart buildings with local battery system and aggregated electric vehicles," Applied Energy, Elsevier, vol. 299(C).
    8. Min-Hwi Kim & Dong-Won Lee & Deuk-Won Kim & Young-Sub An & Jae-Ho Yun, 2021. "Energy Performance Investigation of Bi-Directional Convergence Energy Prosumers for an Energy Sharing Community," Energies, MDPI, vol. 14(17), pages 1-17, September.
    9. Sang-Guk Yum & Kiyoung Son & Seunghyun Son & Ji-Myong Kim, 2020. "Identifying Risk Indicators for Natural Hazard-Related Power Outages as a Component of Risk Assessment: An Analysis Using Power Outage Data from Hurricane Irma," Sustainability, MDPI, vol. 12(18), pages 1-15, September.
    10. Popović, Željko N. & KovaÄ ki, Neven V. & Popović, Dragan S., 2020. "Resilient distribution network planning under the severe windstorms using a risk-based approach," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    11. Chen, Lei & Jiang, Yuqi & Zheng, Shencong & Deng, Xinyi & Chen, Hongkun & Islam, Md. Rabiul, 2023. "A two-layer optimal configuration approach of energy storage systems for resilience enhancement of active distribution networks," Applied Energy, Elsevier, vol. 350(C).
    12. Zeng, Zhiguo & Fang, Yi-Ping & Zhai, Qingqing & Du, Shijia, 2021. "A Markov reward process-based framework for resilience analysis of multistate energy systems under the threat of extreme events," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    13. Alain Aoun & Mehdi Adda & Adrian Ilinca & Mazen Ghandour & Hussein Ibrahim, 2024. "Centralized vs. Decentralized Electric Grid Resilience Analysis Using Leontief’s Input–Output Model," Energies, MDPI, vol. 17(6), pages 1-21, March.
    14. Mishra, Sakshi & Anderson, Kate & Miller, Brian & Boyer, Kyle & Warren, Adam, 2020. "Microgrid resilience: A holistic approach for assessing threats, identifying vulnerabilities, and designing corresponding mitigation strategies," Applied Energy, Elsevier, vol. 264(C).
    15. Khalilullah Mayar & David G. Carmichael & Xuesong Shen, 2022. "Resilience and Systems—A Review," Sustainability, MDPI, vol. 14(14), pages 1-22, July.
    16. Anjie Lu & Jianguo Zhou & Minglei Qin & Danchen Liu, 2024. "Considering Carbon–Hydrogen Coupled Integrated Energy Systems: A Pathway to Sustainable Energy Transition in China Under Uncertainty," Sustainability, MDPI, vol. 16(21), pages 1-32, October.
    17. Liping Huang & Zhaoxiong Huang & Chun Sing Lai & Guangya Yang & Zhuoli Zhao & Ning Tong & Xiaomei Wu & Loi Lei Lai, 2021. "Augmented Power Dispatch for Resilient Operation through Controllable Series Compensation and N-1-1 Contingency Assessment," Energies, MDPI, vol. 14(16), pages 1-24, August.
    18. Ahmadi, Somayeh & Saboohi, Yadollah & Vakili, Ali, 2021. "Frameworks, quantitative indicators, characters, and modeling approaches to analysis of energy system resilience: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    19. Skolfield, J. Kyle & Escobedo, Adolfo R., 2022. "Operations research in optimal power flow: A guide to recent and emerging methodologies and applications," European Journal of Operational Research, Elsevier, vol. 300(2), pages 387-404.
    20. Ghafory-Ashtiany, Mohsen & Arghavani, Mahban, 2022. "Physical performance of power grids against earthquakes: from framework to implementation," International Journal of Critical Infrastructure Protection, Elsevier, vol. 39(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2024:i:20:p:3241-:d:1500246. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.