IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v193y2020ics0360544219323175.html
   My bibliography  Save this article

Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty

Author

Listed:
  • Nyong-Bassey, Bassey Etim
  • Giaouris, Damian
  • Patsios, Charalampos
  • Papadopoulou, Simira
  • Papadopoulos, Athanasios I.
  • Walker, Sara
  • Voutetakis, Spyros
  • Seferlis, Panos
  • Gadoue, Shady

Abstract

Hybrid energy storage systems (HESS) involve synergies between multiple energy storage technologies with complementary operating features aimed at enhancing the reliability of intermittent renewable energy sources (RES). Nevertheless, coordinating HESS through optimized energy management strategies (EMS) introduces complexity. The latter has been previously addressed by the authors through a systems-level graphical EMS via Power Pinch Analysis (PoPA). Although of proven efficiency, accounting for uncertainty with PoPA has been an issue, due to the assumption of a perfect day ahead (DA) generation and load profiles forecast. This paper proposes three adaptive PoPA-based EMS, aimed at negating load demand and RES stochastic variability. Each method has its own merits such as; reduced computational complexity and improved accuracy depending on the probability density function of uncertainty. The first and simplest adaptive scheme is based on a receding horizon model predictive control framework. The second employs a Kalman filter, whereas the third is based on a machine learning algorithm. The three methods are assessed on a real isolated HESS microgrid built in Greece. In validating the proposed methods against the DA PoPA, the proposed methods all performed better with regards to violation of the energy storage operating constraints and plummeting carbon emission footprint.

Suggested Citation

  • Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    • Handle: RePEc:eee:energy:v:193:y:2020:i:c:s0360544219323175
    DOI: 10.1016/j.energy.2019.116622
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544219323175
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2019.116622?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhang, Yan & Fu, Lijun & Zhu, Wanlu & Bao, Xianqiang & Liu, Cang, 2018. "Robust model predictive control for optimal energy management of island microgrids with uncertainties," Energy, Elsevier, vol. 164(C), pages 1229-1241.
    2. Petronilla Fragiacomo & Giuseppe De Lorenzo & Orlando Corigliano, 2018. "Performance Analysis of an Intermediate Temperature Solid Oxide Electrolyzer Test Bench under a CO 2 -H 2 O Feed Stream," Energies, MDPI, vol. 11(9), pages 1-17, August.
    3. Janghorban Esfahani, Iman & Lee, SeungChul & Yoo, ChangKyoo, 2015. "Extended-power pinch analysis (EPoPA) for integration of renewable energy systems with battery/hydrogen storages," Renewable Energy, Elsevier, vol. 80(C), pages 1-14.
    4. Aktas, Ahmet & Erhan, Koray & Özdemir, Sule & Özdemir, Engin, 2018. "Dynamic energy management for photovoltaic power system including hybrid energy storage in smart grid applications," Energy, Elsevier, vol. 162(C), pages 72-82.
    5. Varbanov, Petar Sabev & Fodor, Zsófia & Klemeš, Jiří Jaromír, 2012. "Total Site targeting with process specific minimum temperature difference (ΔTmin)," Energy, Elsevier, vol. 44(1), pages 20-28.
    6. Li, Xunming & Han, Lijin & Liu, Hui & Wang, Weida & Xiang, Changle, 2019. "Real-time optimal energy management strategy for a dual-mode power-split hybrid electric vehicle based on an explicit model predictive control algorithm," Energy, Elsevier, vol. 172(C), pages 1161-1178.
    7. Wieczorek, Maciej & Lewandowski, Mirosław, 2017. "A mathematical representation of an energy management strategy for hybrid energy storage system in electric vehicle and real time optimization using a genetic algorithm," Applied Energy, Elsevier, vol. 192(C), pages 222-233.
    8. Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
    9. Zhao, Haoran & Wu, Qiuwei & Hu, Shuju & Xu, Honghua & Rasmussen, Claus Nygaard, 2015. "Review of energy storage system for wind power integration support," Applied Energy, Elsevier, vol. 137(C), pages 545-553.
    10. Takeda, Hisashi & Tamura, Yoshiyasu & Sato, Seisho, 2016. "Using the ensemble Kalman filter for electricity load forecasting and analysis," Energy, Elsevier, vol. 104(C), pages 184-198.
    11. Olatomiwa, Lanre & Mekhilef, Saad & Ismail, M.S. & Moghavvemi, M., 2016. "Energy management strategies in hybrid renewable energy systems: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 62(C), pages 821-835.
    12. Du, Jiuyu & Zhang, Xiaobin & Wang, Tianze & Song, Ziyou & Yang, Xueqing & Wang, Hewu & Ouyang, Minggao & Wu, Xiaogang, 2018. "Battery degradation minimization oriented energy management strategy for plug-in hybrid electric bus with multi-energy storage system," Energy, Elsevier, vol. 165(PA), pages 153-163.
    13. Cai, Mengting & Huang, Guohe & Chen, Jiapei & Li, Yunhuan & Fan, Yurui, 2018. "A generalized fuzzy chance-constrained energy systems planning model for Guangzhou, China," Energy, Elsevier, vol. 165(PA), pages 191-204.
    14. Tabar, Vahid Sohrabi & Jirdehi, Mehdi Ahmadi & Hemmati, Reza, 2017. "Energy management in microgrid based on the multi objective stochastic programming incorporating portable renewable energy resource as demand response option," Energy, Elsevier, vol. 118(C), pages 827-839.
    15. Bruni, G. & Cordiner, S. & Mulone, V. & Sinisi, V. & Spagnolo, F., 2016. "Energy management in a domestic microgrid by means of model predictive controllers," Energy, Elsevier, vol. 108(C), pages 119-131.
    16. Norbu, Sonam & Bandyopadhyay, Santanu, 2017. "Power Pinch Analysis for optimal sizing of renewable-based isolated system with uncertainties," Energy, Elsevier, vol. 135(C), pages 466-475.
    17. Giaouris, Damian & Papadopoulos, Athanasios I. & Ziogou, Chrysovalantou & Ipsakis, Dimitris & Voutetakis, Spyros & Papadopoulou, Simira & Seferlis, Panos & Stergiopoulos, Fotis & Elmasides, Costas, 2013. "Performance investigation of a hybrid renewable power generation and storage system using systemic power management models," Energy, Elsevier, vol. 61(C), pages 621-635.
    18. Hu, Ming-Che & Lu, Su-Ying & Chen, Yen-Haw, 2016. "Stochastic programming and market equilibrium analysis of microgrids energy management systems," Energy, Elsevier, vol. 113(C), pages 662-670.
    19. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    20. Giaouris, Damian & Papadopoulos, Athanasios I. & Patsios, Charalampos & Walker, Sara & Ziogou, Chrysovalantou & Taylor, Phil & Voutetakis, Spyros & Papadopoulou, Simira & Seferlis, Panos, 2018. "A systems approach for management of microgrids considering multiple energy carriers, stochastic loads, forecasting and demand side response," Applied Energy, Elsevier, vol. 226(C), pages 546-559.
    21. Rocchetta, R. & Bellani, L. & Compare, M. & Zio, E. & Patelli, E., 2019. "A reinforcement learning framework for optimal operation and maintenance of power grids," Applied Energy, Elsevier, vol. 241(C), pages 291-301.
    22. Giaouris, Damian & Papadopoulos, Athanasios I. & Seferlis, Panos & Voutetakis, Spyros & Papadopoulou, Simira, 2016. "Power grand composite curves shaping for adaptive energy management of hybrid microgrids," Renewable Energy, Elsevier, vol. 95(C), pages 433-448.
    23. Liu, Teng & Wang, Bo & Yang, Chenglang, 2018. "Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning," Energy, Elsevier, vol. 160(C), pages 544-555.
    24. Kuznetsova, Elizaveta & Li, Yan-Fu & Ruiz, Carlos & Zio, Enrico & Ault, Graham & Bell, Keith, 2013. "Reinforcement learning for microgrid energy management," Energy, Elsevier, vol. 59(C), pages 133-146.
    25. Hadayeghparast, Shahrzad & SoltaniNejad Farsangi, Alireza & Shayanfar, Heidarali, 2019. "Day-ahead stochastic multi-objective economic/emission operational scheduling of a large scale virtual power plant," Energy, Elsevier, vol. 172(C), pages 630-646.
    26. Dorji, Tshering & Urmee, Tania & Jennings, Philip, 2012. "Options for off-grid electrification in the Kingdom of Bhutan," Renewable Energy, Elsevier, vol. 45(C), pages 51-58.
    27. Wan Alwi, Sharifah Rafidah & Mohammad Rozali, Nor Erniza & Abdul-Manan, Zainuddin & Klemeš, Jiří Jaromír, 2012. "A process integration targeting method for hybrid power systems," Energy, Elsevier, vol. 44(1), pages 6-10.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sabarathinam Srinivasan & Suresh Kumarasamy & Zacharias E. Andreadakis & Pedro G. Lind, 2023. "Artificial Intelligence and Mathematical Models of Power Grids Driven by Renewable Energy Sources: A Survey," Energies, MDPI, vol. 16(14), pages 1-56, July.
    2. He, Yi & Guo, Su & Dong, Peixin & Wang, Chen & Huang, Jing & Zhou, Jianxu, 2022. "Techno-economic comparison of different hybrid energy storage systems for off-grid renewable energy applications based on a novel probabilistic reliability index," Applied Energy, Elsevier, vol. 328(C).
    3. Mazzeo, Domenico & Herdem, Münür Sacit & Matera, Nicoletta & Bonini, Matteo & Wen, John Z. & Nathwani, Jatin & Oliveti, Giuseppe, 2021. "Artificial intelligence application for the performance prediction of a clean energy community," Energy, Elsevier, vol. 232(C).
    4. Zhang, Li & Gao, Yan & Zhu, Hongbo & Tao, Li, 2022. "Bi-level stochastic real-time pricing model in multi-energy generation system: A reinforcement learning approach," Energy, Elsevier, vol. 239(PA).
    5. Chen, Hui & Zhang, Zehui & Guan, Cong & Gao, Haibo, 2020. "Optimization of sizing and frequency control in battery/supercapacitor hybrid energy storage system for fuel cell ship," Energy, Elsevier, vol. 197(C).
    6. Song, Ke & Ding, Yuhang & Hu, Xiao & Xu, Hongjie & Wang, Yimin & Cao, Jing, 2021. "Degradation adaptive energy management strategy using fuel cell state-of-health for fuel economy improvement of hybrid electric vehicle," Applied Energy, Elsevier, vol. 285(C).
    7. Tayerani Charmchi, Amir Saman & Ifaei, Pouya & Yoo, ChangKyoo, 2021. "Smart supply-side management of optimal hydro reservoirs using the water/energy nexus concept: A hydropower pinch analysis," Applied Energy, Elsevier, vol. 281(C).
    8. Totaro, Simone & Boukas, Ioannis & Jonsson, Anders & Cornélusse, Bertrand, 2021. "Lifelong control of off-grid microgrid with model-based reinforcement learning," Energy, Elsevier, vol. 232(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Totaro, Simone & Boukas, Ioannis & Jonsson, Anders & Cornélusse, Bertrand, 2021. "Lifelong control of off-grid microgrid with model-based reinforcement learning," Energy, Elsevier, vol. 232(C).
    2. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).
    3. Zia, Muhammad Fahad & Elbouchikhi, Elhoussin & Benbouzid, Mohamed, 2018. "Microgrids energy management systems: A critical review on methods, solutions, and prospects," Applied Energy, Elsevier, vol. 222(C), pages 1033-1055.
    4. Klemeš, Jiří Jaromír & Varbanov, Petar Sabev & Walmsley, Timothy G. & Jia, Xuexiu, 2018. "New directions in the implementation of Pinch Methodology (PM)," Renewable and Sustainable Energy Reviews, Elsevier, vol. 98(C), pages 439-468.
    5. Giaouris, Damian & Papadopoulos, Athanasios I. & Seferlis, Panos & Voutetakis, Spyros & Papadopoulou, Simira, 2016. "Power grand composite curves shaping for adaptive energy management of hybrid microgrids," Renewable Energy, Elsevier, vol. 95(C), pages 433-448.
    6. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    7. Bizon, Nicu, 2019. "Efficient fuel economy strategies for the Fuel Cell Hybrid Power Systems under variable renewable/load power profile," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    8. Chen, Pengzhan & Liu, Mengchao & Chen, Chuanxi & Shang, Xin, 2019. "A battery management strategy in microgrid for personalized customer requirements," Energy, Elsevier, vol. 189(C).
    9. Giaouris, Damian & Papadopoulos, Athanasios I. & Patsios, Charalampos & Walker, Sara & Ziogou, Chrysovalantou & Taylor, Phil & Voutetakis, Spyros & Papadopoulou, Simira & Seferlis, Panos, 2018. "A systems approach for management of microgrids considering multiple energy carriers, stochastic loads, forecasting and demand side response," Applied Energy, Elsevier, vol. 226(C), pages 546-559.
    10. Norbu, Sonam & Bandyopadhyay, Santanu, 2017. "Power Pinch Analysis for optimal sizing of renewable-based isolated system with uncertainties," Energy, Elsevier, vol. 135(C), pages 466-475.
    11. de la Hoz, Jordi & Martín, Helena & Alonso, Alex & Carolina Luna, Adriana & Matas, José & Vasquez, Juan C. & Guerrero, Josep M., 2019. "Regulatory-framework-embedded energy management system for microgrids: The case study of the Spanish self-consumption scheme," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    12. Zheng, Lingwei & Wu, Hao & Guo, Siqi & Sun, Xinyu, 2023. "Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy," Energy, Elsevier, vol. 277(C).
    13. Yi Kuang & Xiuli Wang & Hongyang Zhao & Yijun Huang & Xianlong Chen & Xifan Wang, 2020. "Agent-Based Energy Sharing Mechanism Using Deep Deterministic Policy Gradient Algorithm," Energies, MDPI, vol. 13(19), pages 1-20, September.
    14. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    15. Tsianikas, Stamatis & Yousefi, Nooshin & Zhou, Jian & Rodgers, Mark D. & Coit, David, 2021. "A storage expansion planning framework using reinforcement learning and simulation-based optimization," Applied Energy, Elsevier, vol. 290(C).
    16. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    17. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    18. Fontenot, Hannah & Dong, Bing, 2019. "Modeling and control of building-integrated microgrids for optimal energy management – A review," Applied Energy, Elsevier, vol. 254(C).
    19. Liu, Yixin & Shi, Haoqi & Guo, Li & Xu, Tao & Zhao, Bo & Wang, Chengshan, 2022. "Towards long-period operational reliability of independent microgrid: A risk-aware energy scheduling and stochastic optimization method," Energy, Elsevier, vol. 254(PB).
    20. Zhu, Junjie & Huang, Shengjun & Liu, Yajie & Lei, Hongtao & Sang, Bo, 2021. "Optimal energy management for grid-connected microgrids via expected-scenario-oriented robust optimization," Energy, Elsevier, vol. 216(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:193:y:2020:i:c:s0360544219323175. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.