IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v285y2021ics0306261920317608.html
   My bibliography  Save this article

Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system

Author

Listed:
  • Li, Jiawen
  • Yu, Tao
  • Zhang, Xiaoshun
  • Li, Fusheng
  • Lin, Dan
  • Zhu, Hanxin

Abstract

To balance the stochastic power disturbance in integrated energy system (IES), a novel automatic generation control (AGC) dispatch is proposed by taking account of the regulation rule that applies to a performance-based frequency regulation market, with the aim to reduce area control deviation and regulation mileage payment while complying with constraints of various regulation units. Thus, a multiple experience pool replay twin delayed deep deterministic policy gradient (MEPR-TD3) is put forward to improve the training efficiency and the action quality via four improvements including the multiple experience pool probability replay strategy. Finally, the performance of the proposed algorithm is verified on an extended two-area load frequency control (LFC) model and Hainan province IES with different demand of multiple energy.

Suggested Citation

  • Li, Jiawen & Yu, Tao & Zhang, Xiaoshun & Li, Fusheng & Lin, Dan & Zhu, Hanxin, 2021. "Efficient experience replay based deep deterministic policy gradient for AGC dispatch in integrated energy system," Applied Energy, Elsevier, vol. 285(C).
    • Handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317608
    DOI: 10.1016/j.apenergy.2020.116386
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920317608
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2020.116386?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhang, Xiaoshun & Chen, Yixuan & Yu, Tao & Yang, Bo & Qu, Kaiping & Mao, Senmao, 2017. "Equilibrium-inspired multiagent optimizer with extreme transfer learning for decentralized optimal carbon-energy combined-flow of large-scale power systems," Applied Energy, Elsevier, vol. 189(C), pages 157-176.
    2. Xi, Lei & Chen, Jianfeng & Huang, Yuehua & Xu, Yanchun & Liu, Lang & Zhou, Yimin & Li, Yudan, 2018. "Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel," Energy, Elsevier, vol. 153(C), pages 977-987.
    3. Bahrami, Saadat & Hooshmand, Rahmat-Allah & Parastegari, Moein, 2014. "Short term electric load forecasting by wavelet transform and grey model improved by PSO (particle swarm optimization) algorithm," Energy, Elsevier, vol. 72(C), pages 434-442.
    4. Zhang, Xiaoshun & Tan, Tian & Yang, Bo & Wang, Jingbo & Li, Shengnan & He, Tingyi & Yang, Lei & Yu, Tao & Sun, Liming, 2020. "Greedy search based data-driven algorithm of centralized thermoelectric generation system under non-uniform temperature distribution," Applied Energy, Elsevier, vol. 260(C).
    5. Yang, Bo & Yu, Tao & Shu, Hongchun & Dong, Jun & Jiang, Lin, 2018. "Robust sliding-mode control of wind energy conversion systems for optimal power extraction via nonlinear perturbation observers," Applied Energy, Elsevier, vol. 210(C), pages 711-723.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shen, Rendong & Zhong, Shengyuan & Wen, Xin & An, Qingsong & Zheng, Ruifan & Li, Yang & Zhao, Jun, 2022. "Multi-agent deep reinforcement learning optimization framework for building energy system with renewable energy," Applied Energy, Elsevier, vol. 312(C).
    2. Ren, Kezheng & Liu, Jun & Liu, Xinglei & Nie, Yongxin, 2023. "Reinforcement Learning-Based Bi-Level strategic bidding model of Gas-fired unit in integrated electricity and natural gas markets preventing market manipulation," Applied Energy, Elsevier, vol. 336(C).
    3. Li, Jiawen & Zhou, Tao, 2023. "Active fault-tolerant coordination energy management for a proton exchange membrane fuel cell using curriculum-based multiagent deep meta-reinforcement learning," Renewable and Sustainable Energy Reviews, Elsevier, vol. 185(C).
    4. Lu, Xin & Qiu, Jing & Zhang, Cuo & Lei, Gang & Zhu, Jianguo, 2024. "Seizing unconventional arbitrage opportunities in virtual power plants: A profitable and flexible recruitment approach," Applied Energy, Elsevier, vol. 358(C).
    5. Hou, Guolian & Huang, Ting & Zheng, Fumeng & Gong, Linjuan & Huang, Congzhi & Zhang, Jianhua, 2023. "Application of multi-agent EADRC in flexible operation of combined heat and power plant considering carbon emission and economy," Energy, Elsevier, vol. 263(PB).
    6. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    7. Mazare, Mahmood, 2024. "Adaptive optimal secure wind power generation control for variable speed wind turbine systems via reinforcement learning," Applied Energy, Elsevier, vol. 353(PA).
    8. Fan, Wei & Tan, Qingbo & Zhang, Amin & Ju, Liwei & Wang, Yuwei & Yin, Zhe & Li, Xudong, 2023. "A Bi-level optimization model of integrated energy system considering wind power uncertainty," Renewable Energy, Elsevier, vol. 202(C), pages 973-991.
    9. Li, Jiawen & Yu, Tao & Zhang, Xiaoshun, 2022. "Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 306(PA).
    10. Li, Bin & Wang, Shuai & Li, Botong & Li, Hongbo & Wu, Jianzhong, 2023. "Optimal performance evaluation of thermal AGC units based on multi-dimensional feature analysis," Applied Energy, Elsevier, vol. 339(C).
    11. Li, Jiawen, 2022. "A multi-objective energy coordinative and management policy for solid oxide fuel cell using triune brain large-scale multi-agent deep deterministic policy gradient," Applied Energy, Elsevier, vol. 324(C).
    12. Kumar Jadoun, Vinay & Rahul Prashanth, G & Suhas Joshi, Siddharth & Narayanan, K. & Malik, Hasmat & García Márquez, Fausto Pedro, 2022. "Optimal fuzzy based economic emission dispatch of combined heat and power units using dynamically controlled Whale Optimization Algorithm," Applied Energy, Elsevier, vol. 315(C).
    13. Li, Jiawen & Zhou, Tao & Keke, He & Yu, Hengwen & Du, Hongwei & Liu, Shuangyu & Cui, Haoyang, 2023. "Distributed quantum multiagent deep meta reinforcement learning for area autonomy energy management of a multiarea microgrid," Applied Energy, Elsevier, vol. 343(C).
    14. Tao, Fazhan & Fu, Zhigao & Gong, Huixian & Ji, Baofeng & Zhou, Yao, 2023. "Twin delayed deep deterministic policy gradient based energy management strategy for fuel cell/battery/ultracapacitor hybrid electric vehicles considering predicted terrain information," Energy, Elsevier, vol. 283(C).
    15. Quan, Yue & Xi, Lei, 2024. "Smart generation system: A decentralized multi-agent control architecture based on improved consensus algorithm for generation command dispatch of sustainable energy systems," Applied Energy, Elsevier, vol. 365(C).
    16. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    17. Li, Jiawen & Yu, Tao & Yang, Bo, 2021. "A data-driven output voltage control of solid oxide fuel cell using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    18. Pengcheng Ni & Zhiyuan Ye & Can Cao & Zhimin Guo & Jian Zhao & Xing He, 2023. "Cooperative Game-Based Collaborative Optimal Regulation-Assisted Digital Twins for Wide-Area Distributed Energy," Energies, MDPI, vol. 16(6), pages 1-17, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    2. Yin, Linfei & Zhang, Bin, 2021. "Time series generative adversarial network controller for long-term smart generation control of microgrids," Applied Energy, Elsevier, vol. 281(C).
    3. Wei Sun & Chongchong Zhang, 2018. "A Hybrid BA-ELM Model Based on Factor Analysis and Similar-Day Approach for Short-Term Load Forecasting," Energies, MDPI, vol. 11(5), pages 1-18, May.
    4. Zhang, Meng & Guo, Huan & Sun, Ming & Liu, Sifeng & Forrest, Jeffrey, 2022. "A novel flexible grey multivariable model and its application in forecasting energy consumption in China," Energy, Elsevier, vol. 239(PE).
    5. Dongxiao Niu & Yi Liang & Wei-Chiang Hong, 2017. "Wind Speed Forecasting Based on EMD and GRNN Optimized by FOA," Energies, MDPI, vol. 10(12), pages 1-18, December.
    6. Lingqin Xia & Guang Chen & Tao Wu & Yu Gao & Ardashir Mohammadzadeh & Ebrahim Ghaderpour, 2022. "Optimal Intelligent Control for Doubly Fed Induction Generators," Mathematics, MDPI, vol. 11(1), pages 1-16, December.
    7. Hannah Jessie Rani R. & Aruldoss Albert Victoire T., 2018. "Training radial basis function networks for wind speed prediction using PSO enhanced differential search optimizer," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-35, May.
    8. Da Liu & Kun Sun & Han Huang & Pingzhou Tang, 2018. "Monthly Load Forecasting Based on Economic Data by Decomposition Integration Theory," Sustainability, MDPI, vol. 10(9), pages 1-22, September.
    9. Hao Hao & Ji Zhang & Qian Zhang & Li Yao & Yichen Sun, 2021. "Improved gray neural network model for healthcare waste recycling forecasting," Journal of Combinatorial Optimization, Springer, vol. 42(4), pages 813-830, November.
    10. Xinyu Han & Rongrong Li, 2019. "Comparison of Forecasting Energy Consumption in East Africa Using the MGM, NMGM, MGM-ARIMA, and NMGM-ARIMA Model," Energies, MDPI, vol. 12(17), pages 1-24, August.
    11. Fan Li & Jingxi Su & Bo Sun, 2023. "An Optimal Scheduling Method for an Integrated Energy System Based on an Improved k-Means Clustering Algorithm," Energies, MDPI, vol. 16(9), pages 1-22, April.
    12. Yin, Linfei & Wang, Tao & Zheng, Baomin, 2021. "Analytical adaptive distributed multi-objective optimization algorithm for optimal power flow problems," Energy, Elsevier, vol. 216(C).
    13. Soudan, Bassel, 2019. "Community-scale baseload generation from marine energy," Energy, Elsevier, vol. 189(C).
    14. Yang, Zhongshan & Wang, Jian, 2018. "A hybrid forecasting approach applied in wind speed forecasting based on a data processing strategy and an optimized artificial intelligence algorithm," Energy, Elsevier, vol. 160(C), pages 87-100.
    15. Ding, Song, 2018. "A novel self-adapting intelligent grey model for forecasting China's natural-gas demand," Energy, Elsevier, vol. 162(C), pages 393-407.
    16. Huazhen Cao & Chong Gao & Xuan He & Yang Li & Tao Yu, 2020. "Multi-Agent Cooperation Based Reduced-Dimension Q(λ) Learning for Optimal Carbon-Energy Combined-Flow," Energies, MDPI, vol. 13(18), pages 1-22, September.
    17. Yang, YouLong & Che, JinXing & Li, YanYing & Zhao, YanJun & Zhu, SuLing, 2016. "An incremental electric load forecasting model based on support vector regression," Energy, Elsevier, vol. 113(C), pages 796-808.
    18. Nana Geng & Yong Zhang & Yixiang Sun & Yunjian Jiang & Dandan Chen, 2015. "Forecasting China’s Annual Biofuel Production Using an Improved Grey Model," Energies, MDPI, vol. 8(10), pages 1-20, October.
    19. Pablo Fernández-Bustamante & Oscar Barambones & Isidro Calvo & Cristian Napole & Mohamed Derbeli, 2021. "Provision of Frequency Response from Wind Farms: A Review," Energies, MDPI, vol. 14(20), pages 1-24, October.
    20. Tuncay Özcan, 2017. "Application of Seasonal and Multivariable Grey Prediction Models for Short-Term Load Forecasting," Alphanumeric Journal, Bahadir Fatih Yildirim, vol. 5(2), pages 329-338, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317608. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.