IDEAS home Printed from https://ideas.repec.org/a/eee/renene/v222y2024ics0960148123018001.html
   My bibliography  Save this article

Intelligent energy management for an on-grid hydrogen refueling station based on dueling double deep Q network algorithm with NoisyNet

Author

Listed:
  • Qi, Yunying
  • Xu, Xiao
  • Liu, Youbo
  • Pan, Li
  • Liu, Junyong
  • Hu, Weihao

Abstract

The development of hydrogen-based vehicles (HVs) can help achieve a zero-carbon future; however, the availability of hydrogen refueling stations (HRSs) prevents their widespread adoption as personal vehicles. Existing studies have investigated the energy management of HRSs using various methods. However, there have been no reports of implementing a deep reinforcement learning (DRL) approach to address these uncertainties and achieve real-time decision making. This study proposes an energy management optimization model of an on-grid HRS based on the improved dueling double deep Q network(D3QN) algorithm with NoisyNet. The primary goal is to reduce the cost of operating an HRS and improve voltage stability while satisfying the hydrogen demand of HVs. Notably, this study adopts an improved version of the double deep Q network (DDQN), that is, the NoisyNet-D3QN (NN-D3QN) approach, because NoisyNet can aid efficient exploration and the dueling network can generalize learning across actions. The adopted NN-D3QN algorithm has better performance than other basic algorithms. Compared with the NN-DDQN, D3QN, and DDQN approaches, the reward of the proposed method increases by 19.08 %, 31.66 %, and 39.26 %, respectively.

Suggested Citation

  • Qi, Yunying & Xu, Xiao & Liu, Youbo & Pan, Li & Liu, Junyong & Hu, Weihao, 2024. "Intelligent energy management for an on-grid hydrogen refueling station based on dueling double deep Q network algorithm with NoisyNet," Renewable Energy, Elsevier, vol. 222(C).
    • Handle: RePEc:eee:renene:v:222:y:2024:i:c:s0960148123018001
    DOI: 10.1016/j.renene.2023.119885
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960148123018001
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.renene.2023.119885?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Niknam, Taher & Azizipanah-Abarghooee, Rasoul & Narimani, Mohammad Rasoul, 2012. "An efficient scenario-based stochastic programming framework for multi-objective optimal micro-grid operation," Applied Energy, Elsevier, vol. 99(C), pages 455-470.
    2. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    3. Won, Wangyun & Kwon, Hweeung & Han, Jee-Hoon & Kim, Jiyong, 2017. "Design and operation of renewable energy sources based hydrogen supply system: Technology integration and optimization," Renewable Energy, Elsevier, vol. 103(C), pages 226-238.
    4. Baziar, Aliasghar & Kavousi-Fard, Abdollah, 2013. "Considering uncertainty in the optimal energy management of renewable micro-grids including storage devices," Renewable Energy, Elsevier, vol. 59(C), pages 158-166.
    5. Zachary P. Cano & Dustin Banham & Siyu Ye & Andreas Hintennach & Jun Lu & Michael Fowler & Zhongwei Chen, 2018. "Batteries and fuel cells for emerging electric vehicle markets," Nature Energy, Nature, vol. 3(4), pages 279-289, April.
    6. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    7. Stoppato, Anna & Cavazzini, Giovanna & Ardizzon, Guido & Rossetti, Antonio, 2014. "A PSO (particle swarm optimization)-based model for the optimal management of a small PV(Photovoltaic)-pump hydro energy storage in a rural dry area," Energy, Elsevier, vol. 76(C), pages 168-174.
    8. Mazzeo, Domenico & Herdem, Münür Sacit & Matera, Nicoletta & Wen, John Z., 2022. "Green hydrogen production: Analysis for different single or combined large-scale photovoltaic and wind renewable systems," Renewable Energy, Elsevier, vol. 200(C), pages 360-378.
    9. Rouholamini, Mehdi & Mohammadian, Mohsen, 2016. "Heuristic-based power management of a grid-connected hybrid energy system combined with hydrogen storage," Renewable Energy, Elsevier, vol. 96(PA), pages 354-365.
    10. Hua, Haochen & Qin, Yuchao & Hao, Chuantong & Cao, Junwei, 2019. "Optimal energy management strategies for energy Internet via deep reinforcement learning approach," Applied Energy, Elsevier, vol. 239(C), pages 598-609.
    11. Pang, Yi & Pan, Lei & Zhang, Jingmei & Chen, Jianwei & Dong, Yan & Sun, Hexu, 2022. "Integrated sizing and scheduling of an off-grid integrated energy system for an isolated renewable energy hydrogen refueling station," Applied Energy, Elsevier, vol. 323(C).
    12. Wu, Xiong & Zhao, Wencheng & Li, Haoyu & Liu, Bingwen & Zhang, Ziyu & Wang, Xiuli, 2021. "Multi-stage stochastic programming based offering strategy for hydrogen fueling station in joint energy, reserve markets," Renewable Energy, Elsevier, vol. 180(C), pages 605-615.
    13. Azaza, Maher & Wallin, Fredrik, 2017. "Multi objective particle swarm optimization of hybrid micro-grid system: A case study in Sweden," Energy, Elsevier, vol. 123(C), pages 108-118.
    14. Li, Chun-Hua & Zhu, Xin-Jian & Cao, Guang-Yi & Sui, Sheng & Hu, Ming-Ruo, 2009. "Dynamic modeling and sizing optimization of stand-alone photovoltaic power systems using hybrid energy storage technology," Renewable Energy, Elsevier, vol. 34(3), pages 815-826.
    15. Troncoso, E. & Lapeña-Rey, N. & Gonzalez, M., 2016. "Design tool for offgrid hydrogen refuelling systems for aerospace applications," Applied Energy, Elsevier, vol. 163(C), pages 476-487.
    16. Lee, Yerim & Hur, Jin, 2019. "A simultaneous approach implementing wind-powered electric vehicle charging stations for charging demand dispersion," Renewable Energy, Elsevier, vol. 144(C), pages 172-179.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Yuyan & Xu, Xiao & Luo, Yichen & Liu, Junyong & Hu, Weihao, 2024. "Distributionally robust planning method for expressway hydrogen refueling station powered by a wind-PV system," Renewable Energy, Elsevier, vol. 225(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhu, Jiaoyiling & Hu, Weihao & Xu, Xiao & Liu, Haoming & Pan, Li & Fan, Haoyang & Zhang, Zhenyuan & Chen, Zhe, 2022. "Optimal scheduling of a wind energy dominated distribution network via a deep reinforcement learning approach," Renewable Energy, Elsevier, vol. 201(P1), pages 792-801.
    2. Qi, Chunyang & Zhu, Yiwen & Song, Chuanxue & Yan, Guangfu & Xiao, Feng & Da wang, & Zhang, Xu & Cao, Jingwei & Song, Shixin, 2022. "Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle," Energy, Elsevier, vol. 238(PA).
    3. Javidsharifi, Mahshid & Niknam, Taher & Aghaei, Jamshid & Mokryani, Geev, 2018. "Multi-objective short-term scheduling of a renewable-based microgrid in the presence of tidal resources and storage devices," Applied Energy, Elsevier, vol. 216(C), pages 367-381.
    4. Shi, Wenzhuo & Huangfu, Yigeng & Xu, Liangcai & Pang, Shengzhao, 2022. "Online energy management strategy considering fuel cell fault for multi-stack fuel cell hybrid vehicle based on multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 328(C).
    5. Mohammed Amine Hoummadi & Manale Bouderbala & Hala Alami Aroussi & Badre Bossoufi & Najib El Ouanjli & Mohammed Karim, 2023. "Survey of Sustainable Energy Sources for Microgrid Energy Management: A Review," Energies, MDPI, vol. 16(7), pages 1-16, March.
    6. Nosratabadi, Seyyed Mostafa & Hooshmand, Rahmat-Allah & Gholipour, Eskandar, 2017. "A comprehensive review on microgrid and virtual power plant concepts employed for distributed energy resources scheduling in power systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 67(C), pages 341-363.
    7. Janghorban Esfahani, Iman & Ifaei, Pouya & Kim, Jinsoo & Yoo, ChangKyoo, 2016. "Design of Hybrid Renewable Energy Systems with Battery/Hydrogen storage considering practical power losses: A MEPoPA (Modified Extended-Power Pinch Analysis)," Energy, Elsevier, vol. 100(C), pages 40-50.
    8. Ruuskanen, Vesa & Koponen, Joonas & Sillanpää, Teemu & Huoman, Kimmo & Kosonen, Antti & Niemelä, Markku & Ahola, Jero, 2018. "Design and implementation of a power-hardware-in-loop simulator for water electrolysis emulation," Renewable Energy, Elsevier, vol. 119(C), pages 106-115.
    9. Fontenot, Hannah & Dong, Bing, 2019. "Modeling and control of building-integrated microgrids for optimal energy management – A review," Applied Energy, Elsevier, vol. 254(C).
    10. Qi, Chunyang & Song, Chuanxue & Xiao, Feng & Song, Shixin, 2022. "Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method," Energy, Elsevier, vol. 250(C).
    11. Olukorede Tijani Adenuga & Senthil Krishnamurthy, 2023. "Economic Power Dispatch of a Grid-Tied Photovoltaic-Based Energy Management System: Co-Optimization Approach," Mathematics, MDPI, vol. 11(15), pages 1-22, July.
    12. Chen, Y. & Trifkovic, M., 2018. "Optimal scheduling of a microgrid in a volatile electricity market environment: Portfolio optimization approach," Applied Energy, Elsevier, vol. 226(C), pages 703-712.
    13. Xu, Xiao & Hu, Weihao & Cao, Di & Liu, Wen & Huang, Qi & Hu, Yanting & Chen, Zhe, 2021. "Enhanced design of an offgrid PV-battery-methanation hybrid energy system for power/gas supply," Renewable Energy, Elsevier, vol. 167(C), pages 440-456.
    14. Boukettaya, Ghada & Krichen, Lotfi, 2014. "A dynamic power management strategy of a grid connected hybrid generation system using wind, photovoltaic and Flywheel Energy Storage System in residential applications," Energy, Elsevier, vol. 71(C), pages 148-159.
    15. Ussama Assad & Muhammad Arshad Shehzad Hassan & Umar Farooq & Asif Kabir & Muhammad Zeeshan Khan & S. Sabahat H. Bukhari & Zain ul Abidin Jaffri & Judit Oláh & József Popp, 2022. "Smart Grid, Demand Response and Optimization: A Critical Review of Computational Methods," Energies, MDPI, vol. 15(6), pages 1-36, March.
    16. Xu, Xiao & Hu, Weihao & Du, Yuefang & Liu, Wen & Liu, Zhou & Huang, Qi & Chen, Zhe, 2020. "Robust chance-constrained gas management for a standalone gas supply system based on wind energy," Energy, Elsevier, vol. 212(C).
    17. Javed, Muhammad Shahzad & Ma, Tao & Jurasz, Jakub & Amin, Muhammad Yasir, 2020. "Solar and wind power generation systems with pumped hydro storage: Review and future perspectives," Renewable Energy, Elsevier, vol. 148(C), pages 176-192.
    18. Rabiee, Abdorreza & Sadeghi, Mohammad & Aghaeic, Jamshid & Heidari, Alireza, 2016. "Optimal operation of microgrids through simultaneous scheduling of electrical vehicles and responsive loads considering wind and PV units uncertainties," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 721-739.
    19. Mazzeo, Domenico & Herdem, Münür Sacit & Matera, Nicoletta & Wen, John Z., 2022. "Green hydrogen production: Analysis for different single or combined large-scale photovoltaic and wind renewable systems," Renewable Energy, Elsevier, vol. 200(C), pages 360-378.
    20. Fazlalipour, Pary & Ehsan, Mehdi & Mohammadi-Ivatloo, Behnam, 2019. "Risk-aware stochastic bidding strategy of renewable micro-grids in day-ahead and real-time markets," Energy, Elsevier, vol. 171(C), pages 689-700.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:renene:v:222:y:2024:i:c:s0960148123018001. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/renewable-energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.