IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i16p3898-d1451610.html
   My bibliography  Save this article

Energy Management System for an Industrial Microgrid Using Optimization Algorithms-Based Reinforcement Learning Technique

Author

Listed:
  • Saugat Upadhyay

    (Faculty of Information Technology, Engineering and Economics, Østfold University College, Kobberslagerstredet 5, 1671 Fredrikstad, Norway)

  • Ibrahim Ahmed

    (Faculty of Information Technology, Engineering and Economics, Østfold University College, Kobberslagerstredet 5, 1671 Fredrikstad, Norway)

  • Lucian Mihet-Popa

    (Faculty of Information Technology, Engineering and Economics, Østfold University College, Kobberslagerstredet 5, 1671 Fredrikstad, Norway)

Abstract

The climate crisis necessitates a global shift to achieve a secure, sustainable, and affordable energy system toward a green energy transition reaching climate neutrality by 2050. Because of this, renewable energy sources have come to the forefront, and the research interest in microgrids that rely on distributed generation and storage systems has exploded. Furthermore, many new markets for energy trading, ancillary services, and frequency reserve markets have provided attractive investment opportunities in exchange for balancing the supply and demand of electricity. Artificial intelligence can be utilized to locally optimize energy consumption, trade energy with the main grid, and participate in these markets. Reinforcement learning (RL) is one of the most promising approaches to achieve this goal because it enables an agent to learn optimal behavior in a microgrid by executing specific actions that maximize the long-term reward signal/function. The study focuses on testing two optimization algorithms: logic-based optimization and reinforcement learning. This paper builds on the existing research framework by combining PPO with machine learning-based load forecasting to produce an optimal solution for an industrial microgrid in Norway under different pricing schemes, including day-ahead pricing and peak pricing. It addresses the peak shaving and price arbitrage challenges by taking the historical data into the algorithm and making the decisions according to the energy consumption pattern, battery characteristics, PV production, and energy price. The RL-based approach is implemented in Python based on real data from the site and in combination with MATLAB-Simulink to validate its results. The application of the RL algorithm achieved an average monthly cost saving of 20% compared with logic-based optimization. These findings contribute to digitalization and decarbonization of energy technology, and support the fundamental goals and policies of the European Green Deal.

Suggested Citation

  • Saugat Upadhyay & Ibrahim Ahmed & Lucian Mihet-Popa, 2024. "Energy Management System for an Industrial Microgrid Using Optimization Algorithms-Based Reinforcement Learning Technique," Energies, MDPI, vol. 17(16), pages 1-18, August.
  • Handle: RePEc:gam:jeners:v:17:y:2024:i:16:p:3898-:d:1451610
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/16/3898/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/17/16/3898/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mughees, Neelam & Jaffery, Mujtaba Hussain & Mughees, Anam & Ansari, Ejaz Ahmad & Mughees, Abdullah, 2023. "Reinforcement learning-based composite differential evolution for integrated demand response scheme in industrial microgrids," Applied Energy, Elsevier, vol. 342(C).
    2. Hussain, Akhtar & Bui, Van-Hai & Kim, Hak-Man, 2019. "Microgrids as a resilience resource and strategies used by microgrids for enhancing resilience," Applied Energy, Elsevier, vol. 240(C), pages 56-72.
    3. Marzband, Mousa & Sumper, Andreas & Ruiz-Álvarez, Albert & Domínguez-García, José Luis & Tomoiagă, Bogdan, 2013. "Experimental evaluation of a real time energy management system for stand-alone microgrids in day-ahead markets," Applied Energy, Elsevier, vol. 106(C), pages 365-376.
    4. Roslan, M.F. & Hannan, M.A. & Ker, Pin Jern & Uddin, M.N., 2019. "Microgrid control methods toward achieving sustainable energy management," Applied Energy, Elsevier, vol. 240(C), pages 583-607.
    5. Roslan, M.F. & Hannan, M.A. & Jern Ker, Pin & Begum, R.A. & Indra Mahlia, TM & Dong, Z.Y., 2021. "Scheduling controller for microgrids energy management system using optimization algorithm in achieving cost saving and emission reduction," Applied Energy, Elsevier, vol. 292(C).
    6. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Seyed Mohammad Sharifhosseini & Taher Niknam & Mohammad Hossein Taabodi & Habib Asadi Aghajari & Ehsan Sheybani & Giti Javidi & Motahareh Pourbehzadi, 2024. "Investigating Intelligent Forecasting and Optimization in Electrical Power Systems: A Comprehensive Review of Techniques and Applications," Energies, MDPI, vol. 17(21), pages 1-35, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Younes Zahraoui & Tarmo Korõtko & Argo Rosin & Saad Mekhilef & Mehdi Seyedmahmoudian & Alex Stojcevski & Ibrahim Alhamrouni, 2024. "AI Applications to Enhance Resilience in Power Systems and Microgrids—A Review," Sustainability, MDPI, vol. 16(12), pages 1-35, June.
    2. Antoine Boche & Clément Foucher & Luiz Fernando Lavado Villa, 2022. "Understanding Microgrid Sustainability: A Systemic and Comprehensive Review," Energies, MDPI, vol. 15(8), pages 1-29, April.
    3. Felix Garcia-Torres & Ascension Zafra-Cabeza & Carlos Silva & Stephane Grieu & Tejaswinee Darure & Ana Estanqueiro, 2021. "Model Predictive Control for Microgrid Functionalities: Review and Future Challenges," Energies, MDPI, vol. 14(5), pages 1-26, February.
    4. Elkholy, M.H. & Metwally, Hamid & Farahat, M.A. & Senjyu, Tomonobu & Elsayed Lotfy, Mohammed, 2022. "Smart centralized energy management system for autonomous microgrid using FPGA," Applied Energy, Elsevier, vol. 317(C).
    5. Raya-Armenta, Jose Maurilio & Bazmohammadi, Najmeh & Avina-Cervantes, Juan Gabriel & Sáez, Doris & Vasquez, Juan C. & Guerrero, Josep M., 2021. "Energy management system optimization in islanded microgrids: An overview and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    6. Anantha Krishnan, V. & Balamurugan, P., 2022. "An efficient DLN2-CRSO approach based dynamic stability enhancement in micro-grid system," Applied Energy, Elsevier, vol. 322(C).
    7. Roslan, M.F. & Hannan, M.A. & Jern Ker, Pin & Begum, R.A. & Indra Mahlia, TM & Dong, Z.Y., 2021. "Scheduling controller for microgrids energy management system using optimization algorithm in achieving cost saving and emission reduction," Applied Energy, Elsevier, vol. 292(C).
    8. Gonzalez-Reina, Antonio Enrique & Garcia-Torres, Felix & Girona-Garcia, Victor & Sanchez-Sanchez-de-Puerta, Alvaro & Jimenez-Romero, F.J. & Jimenez-Hornero, Jorge E., 2024. "Cooperative model predictive control for avoiding critical instants of energy resilience in networked microgrids," Applied Energy, Elsevier, vol. 369(C).
    9. Houben, Nikolaus & Cosic, Armin & Stadler, Michael & Mansoor, Muhammad & Zellinger, Michael & Auer, Hans & Ajanovic, Amela & Haas, Reinhard, 2023. "Optimal dispatch of a multi-energy system microgrid under uncertainty: A renewable energy community in Austria," Applied Energy, Elsevier, vol. 337(C).
    10. Tobajas, Javier & Garcia-Torres, Felix & Roncero-Sánchez, Pedro & Vázquez, Javier & Bellatreche, Ladjel & Nieto, Emilio, 2022. "Resilience-oriented schedule of microgrids with hybrid energy storage system using model predictive control," Applied Energy, Elsevier, vol. 306(PB).
    11. Md Shafiullah & Akib Mostabe Refat & Md Ershadul Haque & Dewan Mabrur Hasan Chowdhury & Md Sanower Hossain & Abdullah G. Alharbi & Md Shafiul Alam & Amjad Ali & Shorab Hossain, 2022. "Review of Recent Developments in Microgrid Energy Management Strategies," Sustainability, MDPI, vol. 14(22), pages 1-30, November.
    12. Hussain Abdalla Sajwani & Bassel Soudan & Abdul Ghani Olabi, 2024. "Empowering Sustainability: Understanding Determinants of Consumer Investment in Microgrid Technology in the UAE," Energies, MDPI, vol. 17(9), pages 1-28, May.
    13. Restrepo, Mauricio & Cañizares, Claudio A. & Simpson-Porco, John W. & Su, Peter & Taruc, John, 2021. "Optimization- and Rule-based Energy Management Systems at the Canadian Renewable Energy Laboratory microgrid facility," Applied Energy, Elsevier, vol. 290(C).
    14. Deng Xu & Yong Long, 2019. "The Impact of Government Subsidy on Renewable Microgrid Investment Considering Double Externalities," Sustainability, MDPI, vol. 11(11), pages 1-15, June.
    15. Saif Jamal & Jagadeesh Pasupuleti & Nur Azzammudin Rahmat & Nadia M. L. Tan, 2022. "Energy Management System for Grid-Connected Nanogrid during COVID-19," Energies, MDPI, vol. 15(20), pages 1-20, October.
    16. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    17. Soheil Mohseni & Alan C. Brent & Daniel Burmester, 2020. "Community Resilience-Oriented Optimal Micro-Grid Capacity Expansion Planning: The Case of Totarabank Eco-Village, New Zealand," Energies, MDPI, vol. 13(15), pages 1-29, August.
    18. Huang, Chunjun & Zong, Yi & You, Shi & Træholt, Chresten & Zheng, Yi & Wang, Jiawei & Zheng, Zixuan & Xiao, Xianyong, 2023. "Economic and resilient operation of hydrogen-based microgrids: An improved MPC-based optimal scheduling scheme considering security constraints of hydrogen facilities," Applied Energy, Elsevier, vol. 335(C).
    19. Zhang, Shizhong & Pei, Wei & Xiao, Hao & Yang, Yanhong & Ye, Hua & Kong, Li, 2020. "Enhancing the survival time of multiple islanding microgrids through composable modular energy router after natural disasters," Applied Energy, Elsevier, vol. 270(C).
    20. Bhowmik, Chiranjib & Bhowmik, Sumit & Ray, Amitava & Pandey, Krishna Murari, 2017. "Optimal green energy planning for sustainable development: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 796-813.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:16:p:3898-:d:1451610. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.