IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v285y2021ics0306261920317566.html
   My bibliography  Save this article

Development and Evaluation of a Smart Charging Strategy for an Electric Vehicle Fleet Based on Reinforcement Learning

Author

Listed:
  • Tuchnitz, Felix
  • Ebell, Niklas
  • Schlund, Jonas
  • Pruckner, Marco

Abstract

Governments are currently subsidizing growth in the electric car market and the associated infrastructure in order to accelerate the transition to more sustainable mobility. To avoid the grid overload that results from simultaneously charging too many electric vehicles, there is a need for smart charging coordination systems. In this paper, we propose a charging coordination system based on Reinforcement Learning using an artificial neural network as a function approximator. Taking into account the baseload present in the power grid, a central agent creates forward-looking, coordinated charging schedules for an electric vehicle fleet of any size. In contrast to optimization-based charging strategies, system dynamics such as future arrivals, departures, and energy consumption do not have to be known beforehand. We implement and compare a range of parameter variants that differ in terms of the reward function and prioritized experience. Subsequently, we use a case study to compare our Reinforcement Learning algorithm with several other charging strategies. The Reinforcement Learning-based charging coordination system is shown to perform very well. All electric vehicles have enough energy for their next trip on departure and charging is carried out almost exclusively during the load valleys at night. Compared with an uncontrolled charging strategy, the Reinforcement Learning algorithm reduces the variance of the total load by 65%. The performance of our Reinforcement Learning concept comes close to that of an optimization-based charging strategy. However, an optimization algorithm needs to know certain information beforehand, such as the vehicle’s departure time and its energy requirement on arriving at the charging station. Our novel Reinforcement Learning-based charging coordination system therefore offers a flexible, easily adaptable, and scalable approach for an electric vehicle fleet under realistic operating conditions.

Suggested Citation

  • Tuchnitz, Felix & Ebell, Niklas & Schlund, Jonas & Pruckner, Marco, 2021. "Development and Evaluation of a Smart Charging Strategy for an Electric Vehicle Fleet Based on Reinforcement Learning," Applied Energy, Elsevier, vol. 285(C).
    • Handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317566
    DOI: 10.1016/j.apenergy.2020.116382
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261920317566
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2020.116382?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hannan, M.A. & Lipu, M.S.H. & Hussain, A. & Mohamed, A., 2017. "A review of lithium-ion battery state of charge estimation and management system in electric vehicle applications: Challenges and recommendations," Renewable and Sustainable Energy Reviews, Elsevier, vol. 78(C), pages 834-854.
    2. Hu, Junjie & Morais, Hugo & Sousa, Tiago & Lind, Morten, 2016. "Electric vehicle fleet management in smart grids: A review of services, optimization and control aspects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 1207-1226.
    3. Monica Alonso & Hortensia Amaris & Jean Gardy Germain & Juan Manuel Galan, 2014. "Optimal Charging Scheduling of Electric Vehicles in Smart Grids by Heuristic Algorithms," Energies, MDPI, vol. 7(4), pages 1-27, April.
    4. Martin Spitzer & Jonas Schlund & Elpiniki Apostolaki-Iosifidou & Marco Pruckner, 2019. "Optimized Integration of Electric Vehicles in Low Voltage Distribution Grids," Energies, MDPI, vol. 12(21), pages 1-19, October.
    5. Yang, Zhile & Li, Kang & Foley, Aoife, 2015. "Computational scheduling methods for integrating plug-in electric vehicles with power systems: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 51(C), pages 396-416.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sajjad Haider & Peter Schegner, 2020. "Heuristic Optimization of Overloading Due to Electric Vehicles in a Low Voltage Grid," Energies, MDPI, vol. 13(22), pages 1-19, November.
    2. Shi You & Junjie Hu & Charalampos Ziras, 2016. "An Overview of Modeling Approaches Applied to Aggregation-Based Fleet Management and Integration of Plug-in Electric Vehicles †," Energies, MDPI, vol. 9(11), pages 1-18, November.
    3. Nezamoddini, Nasim & Wang, Yong, 2016. "Risk management and participation planning of electric vehicles in smart grids for demand response," Energy, Elsevier, vol. 116(P1), pages 836-850.
    4. Barone, G. & Buonomano, A. & Calise, F. & Forzano, C. & Palombo, A., 2019. "Building to vehicle to building concept toward a novel zero energy paradigm: Modelling and case studies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 101(C), pages 625-648.
    5. Vasileios Boglou & Christos-Spyridon Karavas & Konstantinos Arvanitis & Athanasios Karlis, 2020. "A Fuzzy Energy Management Strategy for the Coordination of Electric Vehicle Charging in Low Voltage Distribution Grids," Energies, MDPI, vol. 13(14), pages 1-34, July.
    6. Gaizka Saldaña & Jose Ignacio San Martin & Inmaculada Zamora & Francisco Javier Asensio & Oier Oñederra, 2019. "Electric Vehicle into the Grid: Charging Methodologies Aimed at Providing Ancillary Services Considering Battery Degradation," Energies, MDPI, vol. 12(12), pages 1-37, June.
    7. Weige Zhang & Di Zhang & Biqiang Mu & Le Yi Wang & Yan Bao & Jiuchun Jiang & Hugo Morais, 2017. "Decentralized Electric Vehicle Charging Strategies for Reduced Load Variation and Guaranteed Charge Completion in Regional Distribution Grids," Energies, MDPI, vol. 10(2), pages 1-19, January.
    8. Mahmoudzadeh Andwari, Amin & Pesiridis, Apostolos & Rajoo, Srithar & Martinez-Botas, Ricardo & Esfahanian, Vahid, 2017. "A review of Battery Electric Vehicle technology and readiness levels," Renewable and Sustainable Energy Reviews, Elsevier, vol. 78(C), pages 414-430.
    9. Popović Vlado & Jereb Borut & Kilibarda Milorad & Andrejić Milan & Keshavarzsaleh Abolfazl & Dragan Dejan, 2018. "Electric Vehicles as Electricity Storages in Electric Power Systems," Logistics, Supply Chain, Sustainability and Global Challenges, Sciendo, vol. 9(2), pages 57-72, October.
    10. Sarah Ouédraogo & Ghjuvan Antone Faggianelli & Guillaume Pigelet & Jean Laurent Duchaud & Gilles Notton, 2020. "Application of Optimal Energy Management Strategies for a Building Powered by PV/Battery System in Corsica Island," Energies, MDPI, vol. 13(17), pages 1-20, September.
    11. Adil Amin & Wajahat Ullah Khan Tareen & Muhammad Usman & Haider Ali & Inam Bari & Ben Horan & Saad Mekhilef & Muhammad Asif & Saeed Ahmed & Anzar Mahmood, 2020. "A Review of Optimal Charging Strategy for Electric Vehicles under Dynamic Pricing Schemes in the Distribution Charging Network," Sustainability, MDPI, vol. 12(23), pages 1-28, December.
    12. Essam H. Houssein & Sanchari Deb & Diego Oliva & Hegazy Rezk & Hesham Alhumade & Mokhtar Said, 2021. "Performance of Gradient-Based Optimizer on Charging Station Placement Problem," Mathematics, MDPI, vol. 9(21), pages 1-16, November.
    13. Wu, Juai & Zhang, Mengying & Xu, Tianheng & Gu, Duan & Xie, Dongliang & Zhang, Tengfei & Hu, Honglin & Zhou, Ting, 2023. "A review of key technologies in relation to large-scale clusters of electric vehicles supporting a new power system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 182(C).
    14. Saleh Aghajan-Eshkevari & Sasan Azad & Morteza Nazari-Heris & Mohammad Taghi Ameli & Somayeh Asadi, 2022. "Charging and Discharging of Electric Vehicles in Power Systems: An Updated and Detailed Review of Methods, Control Structures, Objectives, and Optimization Methodologies," Sustainability, MDPI, vol. 14(4), pages 1-31, February.
    15. Vamsi Krishna Reddy, Aala Kalananda & Venkata Lakshmi Narayana, Komanapalli, 2022. "Meta-heuristics optimization in electric vehicles -an extensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 160(C).
    16. Adnane Houbbadi & Rochdi Trigui & Serge Pelissier & Eduardo Redondo-Iglesias & Tanguy Bouton, 2019. "Optimal Scheduling to Manage an Electric Bus Fleet Overnight Charging," Energies, MDPI, vol. 12(14), pages 1-17, July.
    17. Haider, Sajjad & Rizvi, Rida e Zahra & Walewski, John & Schegner, Peter, 2022. "Investigating peer-to-peer power transactions for reducing EV induced network congestion," Energy, Elsevier, vol. 254(PB).
    18. He, Qiang & Yang, Yang & Luo, Chang & Zhai, Jun & Luo, Ronghua & Fu, Chunyun, 2022. "Energy recovery strategy optimization of dual-motor drive electric vehicle based on braking safety and efficient recovery," Energy, Elsevier, vol. 248(C).
    19. Li, Yi & Liu, Kailong & Foley, Aoife M. & Zülke, Alana & Berecibar, Maitane & Nanini-Maury, Elise & Van Mierlo, Joeri & Hoster, Harry E., 2019. "Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 113(C), pages 1-1.
    20. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiao & Fernandez, Carlos, 2023. "A hybrid probabilistic correction model for the state of charge estimation of lithium-ion batteries considering dynamic currents and temperatures," Energy, Elsevier, vol. 273(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:285:y:2021:i:c:s0306261920317566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.