IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v288y2024ics0360544223031195.html
   My bibliography  Save this article

Model-free reinforcement learning-based energy management for plug-in electric vehicles in a cooperative multi-agent home microgrid with consideration of travel behavior

Author

Listed:
  • Salari, Azam
  • Zeinali, Mahdi
  • Marzband, Mousa

Abstract

The rise in popularity of plug-in electric vehicles (PEVs) and the increasing use of renewable energy sources (RESs) have paved the way for advanced energy management systems (EMSs) that optimize energy usage and distribution in home microgrids (H-MGs). This study focuses on the integration of PEVs in H-MGs using an EMS and analyzes its impact on electrical power grids (EPGs). We examine the effect of PEV charging and discharging patterns on H-MG energy flow, considering various scenarios such as high renewable energy generation, different levels of PEV penetration, and EPG connection status. Our findings indicate that an efficient EMS can significantly enhance H-MGs’ overall efficiency by intelligently scheduling PEV charging and discharging, maximizing the use of locally generated renewable energy, and reducing peak load on the EPG. We demonstrate that a cooperative multi-agent system EMS, driven by a robust continuous and real-time fuzzy Q-learning (FQL) method, can reduce electricity market prices by 15% by increasing the use of renewable energy generation by 25%. To fully realize the benefits of EMS, we address challenges such as reducing dependence on EPG by 30%, improving battery state by 12%, and ensuring EPG stability in the face of uncertainties.

Suggested Citation

  • Salari, Azam & Zeinali, Mahdi & Marzband, Mousa, 2024. "Model-free reinforcement learning-based energy management for plug-in electric vehicles in a cooperative multi-agent home microgrid with consideration of travel behavior," Energy, Elsevier, vol. 288(C).
    • Handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223031195
    DOI: 10.1016/j.energy.2023.129725
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544223031195
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2023.129725?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xie, Shaobo & Hu, Xiaosong & Xin, Zongke & Brighton, James, 2019. "Pontryagin’s Minimum Principle based model predictive control of energy management for a plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 236(C), pages 893-905.
    2. Yang, Zhichun & Yang, Fan & Min, Huaidong & Tian, Hao & Hu, Wei & Liu, Jian & Eghbalian, Nasrin, 2023. "Energy management programming to reduce distribution network operating costs in the presence of electric vehicles and renewable energy sources," Energy, Elsevier, vol. 263(PA).
    3. Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
    4. Singh, Bharat & Kumar, Ashwani, 2023. "Optimal energy management and feasibility analysis of hybrid renewable energy sources with BESS and impact of electric vehicle load with demand response program," Energy, Elsevier, vol. 278(PA).
    5. Zhang, Xiaofeng & Kong, Xiaoying & Yan, Renshi & Liu, Yuting & Xia, Peng & Sun, Xiaoqin & Zeng, Rong & Li, Hongqiang, 2023. "Data-driven cooling, heating and electrical load prediction for building integrated with electric vehicles considering occupant travel behavior," Energy, Elsevier, vol. 264(C).
    6. Andre Leippi & Markus Fleschutz & Michael D. Murphy, 2022. "A Review of EV Battery Utilization in Demand Response Considering Battery Degradation in Non-Residential Vehicle-to-Grid Scenarios," Energies, MDPI, vol. 15(9), pages 1-22, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Güven, Aykut Fatih, 2024. "Integrating electric vehicles into hybrid microgrids: A stochastic approach to future-ready renewable energy solutions and management," Energy, Elsevier, vol. 303(C).
    2. Gui, Yonghao & Wei, Baoze & Li, Mingshen & Guerrero, Josep M. & Vasquez, Juan C., 2018. "Passivity-based coordinated control for islanded AC microgrid," Applied Energy, Elsevier, vol. 229(C), pages 551-561.
    3. Johannes Dahlke & Kristina Bogner & Matthias Mueller & Thomas Berger & Andreas Pyka & Bernd Ebersberger, 2020. "Is the Juice Worth the Squeeze? Machine Learning (ML) In and For Agent-Based Modelling (ABM)," Papers 2003.11985, arXiv.org.
    4. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    5. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    6. Xiaohan Fang & Jinkuan Wang & Guanru Song & Yinghua Han & Qiang Zhao & Zhiao Cao, 2019. "Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling," Energies, MDPI, vol. 13(1), pages 1-26, December.
    7. Hao, Ran & Lu, Tianguang & Ai, Qian & Wang, Zhe & Wang, Xiaolong, 2020. "Distributed online learning and dynamic robust standby dispatch for networked microgrids," Applied Energy, Elsevier, vol. 274(C).
    8. Ritter, Andreas & Widmer, Fabio & Duhr, Pol & Onder, Christopher H., 2022. "Long-term stochastic model predictive control for the energy management of hybrid electric vehicles using Pontryagin’s minimum principle and scenario-based optimization," Applied Energy, Elsevier, vol. 322(C).
    9. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    10. Du, Guodong & Zou, Yuan & Zhang, Xudong & Liu, Teng & Wu, Jinlong & He, Dingbo, 2020. "Deep reinforcement learning based energy management for a hybrid electric vehicle," Energy, Elsevier, vol. 201(C).
    11. Anselma, Pier Giuseppe, 2022. "Computationally efficient evaluation of fuel and electrical energy economy of plug-in hybrid electric vehicles with smooth driving constraints," Applied Energy, Elsevier, vol. 307(C).
    12. Guo, Hongqiang & Hou, Daizheng & Du, Shangye & Zhao, Ling & Wu, Jian & Yan, Ning, 2020. "A driving pattern recognition-based energy management for plug-in hybrid electric bus to counter the noise of stochastic vehicle mass," Energy, Elsevier, vol. 198(C).
    13. Ju, Fei & Zhuang, Weichao & Wang, Liangmo & Zhang, Zhe, 2020. "Comparison of four-wheel-drive hybrid powertrain configurations," Energy, Elsevier, vol. 209(C).
    14. Li, J.Y. & Chen, J.J. & Wang, Y.X. & Chen, W.G., 2024. "Combining multi-step reconfiguration with many-objective reduction as iterative bi-level scheduling for stochastic distribution network," Energy, Elsevier, vol. 290(C).
    15. Lingling Hu & Junming Zhou & Feng Jiang & Guangming Xie & Jie Hu & Qinglie Mo, 2023. "Research on Optimization of Valley-Filling Charging for Vehicle Network System Based on Multi-Objective Optimization," Sustainability, MDPI, vol. 16(1), pages 1-25, December.
    16. Pinto, Giuseppe & Piscitelli, Marco Savino & Vázquez-Canteli, José Ramón & Nagy, Zoltán & Capozzoli, Alfonso, 2021. "Coordinated energy management for a cluster of buildings through deep reinforcement learning," Energy, Elsevier, vol. 229(C).
    17. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    18. Esmaeil Ahmadi & Benjamin McLellan & Behnam Mohammadi-Ivatloo & Tetsuo Tezuka, 2020. "The Role of Renewable Energy Resources in Sustainability of Water Desalination as a Potential Fresh-Water Source: An Updated Review," Sustainability, MDPI, vol. 12(13), pages 1-31, June.
    19. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2021. "Real-Time Approximate Equivalent Consumption Minimization Strategy Based on the Single-Shaft Parallel Hybrid Powertrain," Energies, MDPI, vol. 14(23), pages 1-22, November.
    20. Hou, Daizheng & Sun, Qun & Bao, Chunjiang & Cheng, Xingqun & Guo, Hongqiang & Zhao, Ying, 2019. "An all-in-one design method for plug-in hybrid electric buses considering uncertain factor of driving cycles," Applied Energy, Elsevier, vol. 253(C), pages 1-1.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223031195. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.