IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v288y2024ics0360544223031195.html
   My bibliography  Save this article

Model-free reinforcement learning-based energy management for plug-in electric vehicles in a cooperative multi-agent home microgrid with consideration of travel behavior

Author

Listed:
  • Salari, Azam
  • Zeinali, Mahdi
  • Marzband, Mousa

Abstract

The rise in popularity of plug-in electric vehicles (PEVs) and the increasing use of renewable energy sources (RESs) have paved the way for advanced energy management systems (EMSs) that optimize energy usage and distribution in home microgrids (H-MGs). This study focuses on the integration of PEVs in H-MGs using an EMS and analyzes its impact on electrical power grids (EPGs). We examine the effect of PEV charging and discharging patterns on H-MG energy flow, considering various scenarios such as high renewable energy generation, different levels of PEV penetration, and EPG connection status. Our findings indicate that an efficient EMS can significantly enhance H-MGs’ overall efficiency by intelligently scheduling PEV charging and discharging, maximizing the use of locally generated renewable energy, and reducing peak load on the EPG. We demonstrate that a cooperative multi-agent system EMS, driven by a robust continuous and real-time fuzzy Q-learning (FQL) method, can reduce electricity market prices by 15% by increasing the use of renewable energy generation by 25%. To fully realize the benefits of EMS, we address challenges such as reducing dependence on EPG by 30%, improving battery state by 12%, and ensuring EPG stability in the face of uncertainties.

Suggested Citation

  • Salari, Azam & Zeinali, Mahdi & Marzband, Mousa, 2024. "Model-free reinforcement learning-based energy management for plug-in electric vehicles in a cooperative multi-agent home microgrid with consideration of travel behavior," Energy, Elsevier, vol. 288(C).
    • Handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223031195
    DOI: 10.1016/j.energy.2023.129725
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544223031195
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2023.129725?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xie, Shaobo & Hu, Xiaosong & Xin, Zongke & Brighton, James, 2019. "Pontryagin’s Minimum Principle based model predictive control of energy management for a plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 236(C), pages 893-905.
    2. Yang, Zhichun & Yang, Fan & Min, Huaidong & Tian, Hao & Hu, Wei & Liu, Jian & Eghbalian, Nasrin, 2023. "Energy management programming to reduce distribution network operating costs in the presence of electric vehicles and renewable energy sources," Energy, Elsevier, vol. 263(PA).
    3. Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
    4. Singh, Bharat & Kumar, Ashwani, 2023. "Optimal energy management and feasibility analysis of hybrid renewable energy sources with BESS and impact of electric vehicle load with demand response program," Energy, Elsevier, vol. 278(PA).
    5. Zhang, Xiaofeng & Kong, Xiaoying & Yan, Renshi & Liu, Yuting & Xia, Peng & Sun, Xiaoqin & Zeng, Rong & Li, Hongqiang, 2023. "Data-driven cooling, heating and electrical load prediction for building integrated with electric vehicles considering occupant travel behavior," Energy, Elsevier, vol. 264(C).
    6. Andre Leippi & Markus Fleschutz & Michael D. Murphy, 2022. "A Review of EV Battery Utilization in Demand Response Considering Battery Degradation in Non-Residential Vehicle-to-Grid Scenarios," Energies, MDPI, vol. 15(9), pages 1-22, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gui, Yonghao & Wei, Baoze & Li, Mingshen & Guerrero, Josep M. & Vasquez, Juan C., 2018. "Passivity-based coordinated control for islanded AC microgrid," Applied Energy, Elsevier, vol. 229(C), pages 551-561.
    2. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    3. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    4. Hao, Ran & Lu, Tianguang & Ai, Qian & Wang, Zhe & Wang, Xiaolong, 2020. "Distributed online learning and dynamic robust standby dispatch for networked microgrids," Applied Energy, Elsevier, vol. 274(C).
    5. Du, Guodong & Zou, Yuan & Zhang, Xudong & Liu, Teng & Wu, Jinlong & He, Dingbo, 2020. "Deep reinforcement learning based energy management for a hybrid electric vehicle," Energy, Elsevier, vol. 201(C).
    6. Anselma, Pier Giuseppe, 2022. "Computationally efficient evaluation of fuel and electrical energy economy of plug-in hybrid electric vehicles with smooth driving constraints," Applied Energy, Elsevier, vol. 307(C).
    7. Li, J.Y. & Chen, J.J. & Wang, Y.X. & Chen, W.G., 2024. "Combining multi-step reconfiguration with many-objective reduction as iterative bi-level scheduling for stochastic distribution network," Energy, Elsevier, vol. 290(C).
    8. Lingling Hu & Junming Zhou & Feng Jiang & Guangming Xie & Jie Hu & Qinglie Mo, 2023. "Research on Optimization of Valley-Filling Charging for Vehicle Network System Based on Multi-Objective Optimization," Sustainability, MDPI, vol. 16(1), pages 1-25, December.
    9. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    10. Esmaeil Ahmadi & Benjamin McLellan & Behnam Mohammadi-Ivatloo & Tetsuo Tezuka, 2020. "The Role of Renewable Energy Resources in Sustainability of Water Desalination as a Potential Fresh-Water Source: An Updated Review," Sustainability, MDPI, vol. 12(13), pages 1-31, June.
    11. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2021. "Real-Time Approximate Equivalent Consumption Minimization Strategy Based on the Single-Shaft Parallel Hybrid Powertrain," Energies, MDPI, vol. 14(23), pages 1-22, November.
    12. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    13. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    14. Pinto, Giuseppe & Deltetto, Davide & Capozzoli, Alfonso, 2021. "Data-driven district energy management with surrogate models and deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    15. Qi, Chunyang & Zhu, Yiwen & Song, Chuanxue & Yan, Guangfu & Xiao, Feng & Da wang, & Zhang, Xu & Cao, Jingwei & Song, Shixin, 2022. "Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle," Energy, Elsevier, vol. 238(PA).
    16. Jicheng Fang & Yifei Wang & Zhen Lei & Qingshan Xu, 2022. "Control Strategy and Performance Analysis of Electrochemical Energy Storage Station Participating in Power System Frequency Regulation: A Case Study of the Jiangsu Power Grid," Sustainability, MDPI, vol. 14(15), pages 1-31, July.
    17. Manfren, Massimiliano & Nastasi, Benedetto, 2023. "Interpretable data-driven building load profiles modelling for Measurement and Verification 2.0," Energy, Elsevier, vol. 283(C).
    18. Tri Cuong Do & Hoai Vu Anh Truong & Hoang Vu Dao & Cong Minh Ho & Xuan Dinh To & Tri Dung Dang & Kyoung Kwan Ahn, 2019. "Energy Management Strategy of a PEM Fuel Cell Excavator with a Supercapacitor/Battery Hybrid Power Source," Energies, MDPI, vol. 12(22), pages 1-24, November.
    19. Sun, Wenjing & Zou, Yuan & Zhang, Xudong & Guo, Ningyuan & Zhang, Bin & Du, Guodong, 2022. "High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning," Energy, Elsevier, vol. 258(C).
    20. Zheng, Lingwei & Wu, Hao & Guo, Siqi & Sun, Xinyu, 2023. "Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy," Energy, Elsevier, vol. 277(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:288:y:2024:i:c:s0360544223031195. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.