IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v285y2021ics0306261920317608.html
   My bibliography  Save this article

Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system

Author

Listed:
  • Li, Jiawen
  • Yu, Tao
  • Zhang, Xiaoshun
  • Li, Fusheng
  • Lin, Dan
  • Zhu, Hanxin

Abstract

To balance the stochastic power disturbance in integrated energy system (IES), a novel automatic generation control (AGC) dispatch is proposed by taking account of the regulation rule that applies to a performance-based frequency regulation market, with the aim to reduce area control deviation and regulation mileage payment while complying with constraints of various regulation units. Thus, a multiple experience pool replay twin delayed deep deterministic policy gradient (MEPR-TD3) is put forward to improve the training efficiency and the action quality via four improvements including the multiple experience pool probability replay strategy. Finally, the performance of the proposed algorithm is verified on an extended two-area load frequency control (LFC) model and Hainan province IES with different demand of multiple energy.

Suggested Citation

  • Li, Jiawen & Yu, Tao & Zhang, Xiaoshun & Li, Fusheng & Lin, Dan & Zhu, Hanxin, 2021. "Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system," Applied Energy, Elsevier, vol. 285(C).
    • Handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317608
    DOI: 10.1016/j.apenergy.2020.116386
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920317608
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2020.116386?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bahrami, Saadat & Hooshmand, Rahmat-Allah & Parastegari, Moein, 2014. "Short term electric load forecasting by wavelet transform and grey model improved by PSO (particle swarm optimization) algorithm," Energy, Elsevier, vol. 72(C), pages 434-442.
    2. Zhang, Xiaoshun & Chen, Yixuan & Yu, Tao & Yang, Bo & Qu, Kaiping & Mao, Senmao, 2017. "Equilibrium-inspired multiagent optimizer with extreme transfer learning for decentralized optimal carbon-energy combined-flow of large-scale power systems," Applied Energy, Elsevier, vol. 189(C), pages 157-176.
    3. Xi, Lei & Chen, Jianfeng & Huang, Yuehua & Xu, Yanchun & Liu, Lang & Zhou, Yimin & Li, Yudan, 2018. "Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel," Energy, Elsevier, vol. 153(C), pages 977-987.
    4. Zhang, Xiaoshun & Tan, Tian & Yang, Bo & Wang, Jingbo & Li, Shengnan & He, Tingyi & Yang, Lei & Yu, Tao & Sun, Liming, 2020. "Greedy search based data-driven algorithm of centralized thermoelectric generation system under non-uniform temperature distribution," Applied Energy, Elsevier, vol. 260(C).
    5. Yang, Bo & Yu, Tao & Shu, Hongchun & Dong, Jun & Jiang, Lin, 2018. "Robust sliding-mode control of wind energy conversion systems for optimal power extraction via nonlinear perturbation observers," Applied Energy, Elsevier, vol. 210(C), pages 711-723.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shen, Rendong & Zhong, Shengyuan & Wen, Xin & An, Qingsong & Zheng, Ruifan & Li, Yang & Zhao, Jun, 2022. "Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy," Applied Energy, Elsevier, vol. 312(C).
    2. Ren, Kezheng & Liu, Jun & Liu, Xinglei & Nie, Yongxin, 2023. "Reinforcement Learning-Based Bi-Level strategic bidding model of Gas-fired unit in integrated electricity and natural gas markets preventing market manipulation," Applied Energy, Elsevier, vol. 336(C).
    3. Lu, Xin & Qiu, Jing & Zhang, Cuo & Lei, Gang & Zhu, Jianguo, 2024. "Seizing unconventional arbitrage opportunities in virtual power plants: A profitable and flexible recruitment approach," Applied Energy, Elsevier, vol. 358(C).
    4. Hou, Guolian & Huang, Ting & Zheng, Fumeng & Gong, Linjuan & Huang, Congzhi & Zhang, Jianhua, 2023. "Application of multi-agent EADRC in flexible operation of combined heat and power plant considering carbon emission and economy," Energy, Elsevier, vol. 263(PB).
    5. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    6. Li, Jiawen & Yu, Tao & Zhang, Xiaoshun, 2022. "Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 306(PA).
    7. Li, Bin & Wang, Shuai & Li, Botong & Li, Hongbo & Wu, Jianzhong, 2023. "Optimal performance evaluation of thermal AGC units based on multi-dimensional feature analysis," Applied Energy, Elsevier, vol. 339(C).
    8. Li, Jiawen & Zhou, Tao & Keke, He & Yu, Hengwen & Du, Hongwei & Liu, Shuangyu & Cui, Haoyang, 2023. "Distributed quantum multiagent deep meta reinforcement learning for area autonomy energy management of a multiarea microgrid," Applied Energy, Elsevier, vol. 343(C).
    9. Li, Jiawen & Yu, Tao & Yang, Bo, 2021. "A data-driven output voltage control of solid oxide fuel cell using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    10. Li, Jiawen & Zhou, Tao, 2023. "Active fault-tolerant coordination energy management for a proton exchange membrane fuel cell using curriculum-based multiagent deep meta-reinforcement learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 185(C).
    11. Mazare, Mahmood, 2024. "Adaptive optimal secure wind power generation control for variable speed wind turbine systems via reinforcement learning," Applied Energy, Elsevier, vol. 353(PA).
    12. Fan, Wei & Tan, Qingbo & Zhang, Amin & Ju, Liwei & Wang, Yuwei & Yin, Zhe & Li, Xudong, 2023. "A Bi-level optimization model of integrated energy system considering wind power uncertainty," Renewable Energy, Elsevier, vol. 202(C), pages 973-991.
    13. Li, Jiawen, 2022. "A multi-objective energy coordinative and management policy for solid oxide fuel cell using triune brain large-scale multi-agent deep deterministic policy gradient," Applied Energy, Elsevier, vol. 324(C).
    14. Kumar Jadoun, Vinay & Rahul Prashanth, G & Suhas Joshi, Siddharth & Narayanan, K. & Malik, Hasmat & García Márquez, Fausto Pedro, 2022. "Optimal fuzzy based economic emission dispatch of combined heat and power units using dynamically controlled Whale Optimization Algorithm," Applied Energy, Elsevier, vol. 315(C).
    15. Tao, Fazhan & Fu, Zhigao & Gong, Huixian & Ji, Baofeng & Zhou, Yao, 2023. "Twin delayed deep deterministic policy gradient based energy management strategy for fuel cell/battery/ultracapacitor hybrid electric vehicles considering predicted terrain information," Energy, Elsevier, vol. 283(C).
    16. Quan, Yue & Xi, Lei, 2024. "Smart generation system: A decentralized multi-agent control architecture based on improved consensus algorithm for generation command dispatch of sustainable energy systems," Applied Energy, Elsevier, vol. 365(C).
    17. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    18. Pengcheng Ni & Zhiyuan Ye & Can Cao & Zhimin Guo & Jian Zhao & Xing He, 2023. "Cooperative Game-Based Collaborative Optimal Regulation-Assisted Digital Twins for Wide-Area Distributed Energy," Energies, MDPI, vol. 16(6), pages 1-17, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    2. Yin, Linfei & Zhang, Bin, 2021. "Time series generative adversarial network controller for long-term smart generation control of microgrids," Applied Energy, Elsevier, vol. 281(C).
    3. Wei Sun & Chongchong Zhang, 2018. "A Hybrid BA-ELM Model Based on Factor Analysis and Similar-Day Approach for Short-Term Load Forecasting," Energies, MDPI, vol. 11(5), pages 1-18, May.
    4. Zhang, Jiyuan & Tang, Hailong & Chen, Min, 2019. "Linear substitute model-based uncertainty analysis of complicated non-linear energy system performance (case study of an adaptive cycle engine)," Applied Energy, Elsevier, vol. 249(C), pages 87-108.
    5. Zhang, Meng & Guo, Huan & Sun, Ming & Liu, Sifeng & Forrest, Jeffrey, 2022. "A novel flexible grey multivariable model and its application in forecasting energy consumption in China," Energy, Elsevier, vol. 239(PE).
    6. Zhang, Ning & Li, Zhiying & Zou, Xun & Quiring, Steven M., 2019. "Comparison of three short-term load forecast models in Southern California," Energy, Elsevier, vol. 189(C).
    7. Dongxiao Niu & Yi Liang & Wei-Chiang Hong, 2017. "Wind Speed Forecasting Based on EMD and GRNN Optimized by FOA," Energies, MDPI, vol. 10(12), pages 1-18, December.
    8. Lingqin Xia & Guang Chen & Tao Wu & Yu Gao & Ardashir Mohammadzadeh & Ebrahim Ghaderpour, 2022. "Optimal Intelligent Control for Doubly Fed Induction Generators," Mathematics, MDPI, vol. 11(1), pages 1-16, December.
    9. Hongchun Shu & Na An & Bo Yang & Yue Dai & Yu Guo, 2020. "Single Pole-to-Ground Fault Analysis of MMC-HVDC Transmission Lines Based on Capacitive Fuzzy Identification Algorithm," Energies, MDPI, vol. 13(2), pages 1-18, January.
    10. Hannah Jessie Rani R. & Aruldoss Albert Victoire T., 2018. "Training radial basis function networks for wind speed prediction using PSO enhanced differential search optimizer," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-35, May.
    11. Hongchun Shu & Yiming Han & Ran Huang & Yutao Tang & Pulin Cao & Bo Yang & Yu Zhang, 2020. "Fault Model and Travelling Wave Matching Based Single Terminal Fault Location Algorithm for T-Connection Transmission Line: A Yunnan Power Grid Study," Energies, MDPI, vol. 13(6), pages 1-22, March.
    12. Da Liu & Kun Sun & Han Huang & Pingzhou Tang, 2018. "Monthly Load Forecasting Based on Economic Data by Decomposition Integration Theory," Sustainability, MDPI, vol. 10(9), pages 1-22, September.
    13. Hao Hao & Ji Zhang & Qian Zhang & Li Yao & Yichen Sun, 2021. "Improved gray neural network model for healthcare waste recycling forecasting," Journal of Combinatorial Optimization, Springer, vol. 42(4), pages 813-830, November.
    14. Jiang Zeng & Lin Yang & Yuchang Ling & Haoping Chen & Zhonglong Huang & Tao Yu & Bo Yang, 2018. "Smoothly Transitive Fixed Frequency Hysteresis Current Control Based on Optimal Voltage Space Vector," Energies, MDPI, vol. 11(7), pages 1-20, July.
    15. Xinyu Han & Rongrong Li, 2019. "Comparison of Forecasting Energy Consumption in East Africa Using the MGM, NMGM, MGM-ARIMA, and NMGM-ARIMA Model," Energies, MDPI, vol. 12(17), pages 1-24, August.
    16. Hui, Jiuwu & Lee, Yi-Kuen & Yuan, Jingqi, 2023. "Load following control of a PWR with load-dependent parameters and perturbations via fixed-time fractional-order sliding mode and disturbance observer techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 184(C).
    17. Qu, Zongxi & Mao, Wenqian & Zhang, Kequan & Zhang, Wenyu & Li, Zhipeng, 2019. "Multi-step wind speed forecasting based on a hybrid decomposition technique and an improved back-propagation neural network," Renewable Energy, Elsevier, vol. 133(C), pages 919-929.
    18. Fan Li & Jingxi Su & Bo Sun, 2023. "An Optimal Scheduling Method for an Integrated Energy System Based on an Improved k-Means Clustering Algorithm," Energies, MDPI, vol. 16(9), pages 1-22, April.
    19. Umme Mumtahina & Sanath Alahakoon & Peter Wolfs, 2024. "Hyperparameter Tuning of Load-Forecasting Models Using Metaheuristic Optimization Algorithms—A Systematic Review," Mathematics, MDPI, vol. 12(21), pages 1-51, October.
    20. Yin, Linfei & Wang, Tao & Zheng, Baomin, 2021. "Analytical adaptive distributed multi-objective optimization algorithm for optimal power flow problems," Energy, Elsevier, vol. 216(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317608. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.