IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v219y2018icp53-67.html
   My bibliography  Save this article

Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids

Author

Listed:
  • Kofinas, P.
  • Dounis, A.I.
  • Vouros, G.A.

Abstract

This study proposes a cooperative multi-agent system for managing the energy of a stand-alone microgrid. The multi-agent system learns to control the components of the microgrid so as this to achieve its purposes and operate effectively, by means of a distributed, collaborative reinforcement learning method in continuous actions-states space. Stand-alone microgrids present challenges regarding guaranteeing electricity supply and increasing the reliability of the system under the uncertainties introduced by the renewable power sources and the stochastic demand of the consumers. In this article we consider a microgrid that consists of power production, power consumption and power storage units: the power production group includes a Photovoltaic source, a fuel cell and a diesel generator; the power consumption group includes an electrolyzer unit, a desalination plant and a variable electrical load that represent the power consumption of a building; the power storage group includes only the Battery bank. We conjecture that a distributed multi-agent system presents specific advantages to control the microgrid components which operate in a continuous states and actions space: For this purpose we propose the use of fuzzy Q-Learning methods for agents representing microgrid components to act as independent learners, while sharing state variables to coordinate their behavior. Experimental results highlight both the effectiveness of individual agents to control system components, as well as the effectiveness of the multi-agent system to guarantee electricity supply and increase the reliability of the microgrid.

Suggested Citation

  • Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
  • Handle: RePEc:eee:appene:v:219:y:2018:i:c:p:53-67
    DOI: 10.1016/j.apenergy.2018.03.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261918303465
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2018.03.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anvari-Moghaddam, Amjad & Rahimi-Kian, Ashkan & Mirian, Maryam S. & Guerrero, Josep M., 2017. "A multi-agent based energy management solution for integrated buildings and microgrid system," Applied Energy, Elsevier, vol. 203(C), pages 41-56.
    2. Skarvelis-Kazakos, Spyros & Papadopoulos, Panagiotis & Grau Unda, Iñaki & Gorman, Terry & Belaidi, Abdelhafid & Zigan, Stefan, 2016. "Multiple energy carrier optimisation with intelligent agents," Applied Energy, Elsevier, vol. 167(C), pages 323-335.
    3. Hak-Man Kim & Yujin Lim & Tetsuo Kinoshita, 2012. "An Intelligent Multiagent System for Autonomous Microgrid Operation," Energies, MDPI, vol. 5(9), pages 1-16, September.
    4. Xydas, Erotokritos & Marmaras, Charalampos & Cipcigan, Liana M., 2016. "A multi-agent based scheduling algorithm for adaptive electric vehicles charging," Applied Energy, Elsevier, vol. 177(C), pages 354-365.
    5. Kyriakarakos, George & Piromalis, Dimitrios D. & Dounis, Anastasios I. & Arvanitis, Konstantinos G. & Papadakis, George, 2013. "Intelligent demand side energy management system for autonomous polygeneration microgrids," Applied Energy, Elsevier, vol. 103(C), pages 39-51.
    6. El-Sharafy, M. Zaki & Farag, Hany E.Z., 2017. "Back-feed power restoration using distributed constraint optimization in smart distribution grids clustered into microgrids," Applied Energy, Elsevier, vol. 206(C), pages 1102-1117.
    7. Riverso, Stefano & Tucci, Michele & Vasquez, Juan C. & Guerrero, Josep M. & Ferrari-Trecate, Giancarlo, 2018. "Stabilizing plug-and-play regulators and secondary coordinated control for AC islanded microgrids with bus-connected topology," Applied Energy, Elsevier, vol. 210(C), pages 914-924.
    8. Makrygiorgou, Despoina I. & Alexandridis, Antonio T., 2018. "Distributed stabilizing modular control for stand-alone microgrids," Applied Energy, Elsevier, vol. 210(C), pages 925-935.
    9. Zhao, Bo & Xue, Meidong & Zhang, Xuesong & Wang, Caisheng & Zhao, Junhui, 2015. "An MAS based energy management system for a stand-alone microgrid at high altitude," Applied Energy, Elsevier, vol. 143(C), pages 251-261.
    10. Coelho, Vitor N. & Weiss Cohen, Miri & Coelho, Igor M. & Liu, Nian & Guimarães, Frederico Gadelha, 2017. "Multi-agent systems applied for energy systems integration: State-of-the-art applications and trends in microgrids," Applied Energy, Elsevier, vol. 187(C), pages 820-832.
    11. Dounis, A.I. & Caraiscos, C., 2009. "Advanced control systems engineering for energy and comfort management in a building environment--A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(6-7), pages 1246-1261, August.
    12. Wang, Zhu & Wang, Lingfeng & Dounis, Anastasios I. & Yang, Rui, 2012. "Multi-agent control system with information fusion based comfort model for smart buildings," Applied Energy, Elsevier, vol. 99(C), pages 247-254.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ruiqiu Yao & Yukun Hu & Liz Varga, 2023. "Applications of Agent-Based Methods in Multi-Energy Systems—A Systematic Literature Review," Energies, MDPI, vol. 16(5), pages 1-36, March.
    2. Lin, Haiyang & Liu, Yiling & Sun, Qie & Xiong, Rui & Li, Hailong & Wennersten, Ronald, 2018. "The impact of electric vehicle penetration and charging patterns on the management of energy hub – A multi-agent system simulation," Applied Energy, Elsevier, vol. 230(C), pages 189-206.
    3. Fontenot, Hannah & Dong, Bing, 2019. "Modeling and control of building-integrated microgrids for optimal energy management – A review," Applied Energy, Elsevier, vol. 254(C).
    4. Bünning, Felix & Sangi, Roozbeh & Müller, Dirk, 2017. "A Modelica library for the agent-based control of building energy systems," Applied Energy, Elsevier, vol. 193(C), pages 52-59.
    5. Lopez-Rodriguez, I. & Hernandez-Tejera, M., 2015. "Infrastructure based on supernodes and software agents for the implementation of energy markets in demand-response programs," Applied Energy, Elsevier, vol. 158(C), pages 1-11.
    6. He, Senyu & Yin, Jianhua & Zhang, Bin & Wang, Zhaohua, 2018. "How to upgrade an enterprise’s low-carbon technologies under a carbon tax: The trade-off between tax and upgrade fee," Applied Energy, Elsevier, vol. 227(C), pages 564-573.
    7. Gui, Yonghao & Wei, Baoze & Li, Mingshen & Guerrero, Josep M. & Vasquez, Juan C., 2018. "Passivity-based coordinated control for islanded AC microgrid," Applied Energy, Elsevier, vol. 229(C), pages 551-561.
    8. Muhammad Fayaz & DoHyeun Kim, 2018. "Energy Consumption Optimization and User Comfort Management in Residential Buildings Using a Bat Algorithm and Fuzzy Logic," Energies, MDPI, vol. 11(1), pages 1-22, January.
    9. Zhao, Bo & Xue, Meidong & Zhang, Xuesong & Wang, Caisheng & Zhao, Junhui, 2015. "An MAS based energy management system for a stand-alone microgrid at high altitude," Applied Energy, Elsevier, vol. 143(C), pages 251-261.
    10. Korkas, Christos D. & Baldi, Simone & Michailidis, Iakovos & Kosmatopoulos, Elias B., 2015. "Intelligent energy and thermal comfort management in grid-connected microgrids with heterogeneous occupancy schedule," Applied Energy, Elsevier, vol. 149(C), pages 194-203.
    11. Sahoo, Subham & Pullaguram, Deepak & Mishra, Sukumar & Wu, Jianzhong & Senroy, Nilanjan, 2018. "A containment based distributed finite-time controller for bounded voltage regulation & proportionate current sharing in DC microgrids," Applied Energy, Elsevier, vol. 228(C), pages 2526-2538.
    12. Zia, Muhammad Fahad & Elbouchikhi, Elhoussin & Benbouzid, Mohamed, 2018. "Microgrids energy management systems: A critical review on methods, solutions, and prospects," Applied Energy, Elsevier, vol. 222(C), pages 1033-1055.
    13. Lee, J. & Razeghi, G. & Samuelsen, S., 2022. "Generic microgrid controller with self-healing capabilities," Applied Energy, Elsevier, vol. 308(C).
    14. Md Mainul Islam & Mahmood Nagrial & Jamal Rizk & Ali Hellany, 2021. "General Aspects, Islanding Detection, and Energy Management in Microgrids: A Review," Sustainability, MDPI, vol. 13(16), pages 1-45, August.
    15. Shobole, Abdulfetah Abdela & Wadi, Mohammed, 2021. "Multiagent systems application for the smart grid protection," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    16. Halhoul Merabet, Ghezlane & Essaaidi, Mohamed & Ben Haddou, Mohamed & Qolomany, Basheer & Qadir, Junaid & Anan, Muhammad & Al-Fuqaha, Ala & Abid, Mohamed Riduan & Benhaddou, Driss, 2021. "Intelligent building control systems for thermal comfort and energy-efficiency: A systematic review of artificial intelligence-assisted techniques," Renewable and Sustainable Energy Reviews, Elsevier, vol. 144(C).
    17. Babak Arbab-Zavar & Emilio J. Palacios-Garcia & Juan C. Vasquez & Josep M. Guerrero, 2021. "Message Queuing Telemetry Transport Communication Infrastructure for Grid-Connected AC Microgrids Management," Energies, MDPI, vol. 14(18), pages 1-31, September.
    18. Lv, Tianguang & Ai, Qian, 2016. "Interactive energy management of networked microgrids-based active distribution system considering large-scale integration of renewable energy resources," Applied Energy, Elsevier, vol. 163(C), pages 408-422.
    19. Howell, Shaun & Rezgui, Yacine & Hippolyte, Jean-Laurent & Jayan, Bejay & Li, Haijiang, 2017. "Towards the next generation of smart grids: Semantic and holonic multi-agent management of distributed energy resources," Renewable and Sustainable Energy Reviews, Elsevier, vol. 77(C), pages 193-214.
    20. Baloch, Ashfaque Ahmed & Shaikh, Pervez Hameed & Shaikh, Faheemullah & Leghari, Zohaib Hussain & Mirjat, Nayyar Hussain & Uqaili, Muhammad Aslam, 2018. "Simulation tools application for artificial lighting in buildings," Renewable and Sustainable Energy Reviews, Elsevier, vol. 82(P3), pages 3007-3026.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:219:y:2018:i:c:p:53-67. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.