IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i21p5442-d1511057.html
   My bibliography  Save this article

Reinforcement Learning for EV Fleet Smart Charging with On-Site Renewable Energy Sources

Author

Listed:
  • Handong Li

    (Department of Biochemical Engineering, University College London, London WC1H 0AW, UK)

  • Xuewu Dai

    (Mathematics, Physics and Electrical Engineering, Northumbria Univerisity, Newcastle upon Tyne NE1 8ST, UK)

  • Stephen Goldrick

    (Department of Biochemical Engineering, University College London, London WC1H 0AW, UK)

  • Richard Kotter

    (Mathematics, Physics and Electrical Engineering, Northumbria Univerisity, Newcastle upon Tyne NE1 8ST, UK)

  • Nauman Aslam

    (Mathematics, Physics and Electrical Engineering, Northumbria Univerisity, Newcastle upon Tyne NE1 8ST, UK)

  • Saleh Ali

    (School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK)

Abstract

In 2020, the transportation sector was the second largest source of carbon emissions in the UK and in Newcastle upon Tyne, responsible for about 33% of total emissions. To support the UK’s target of reaching net zero emissions by 2050, electric vehicles (EVs) are pivotal in advancing carbon-neutral road transportation. Optimal EV charging requires a better understanding of the unpredictable output from on-site renewable energy sources (ORES). This paper proposes an integrated EV fleet charging schedule with a proximal policy optimization method based on a framework for deep reinforcement learning. For the design of the reinforcement learning environment, mathematical models of wind and solar power generation are created. In addition, the multivariate Gaussian distributions derived from historical weather and EV fleet charging data are utilized to simulate weather and charging demand uncertainty in order to create large datasets for training the model. The optimization problem is expressed as a Markov decision process (MDP) with operational constraints. For training artificial neural networks (ANNs) through successive transition simulations, a proximal policy optimization (PPO) approach is devised. The optimization approach is deployed and evaluated on a real-world scenario comprised of council EV fleet charging data from Leicester, UK. The results show that due to the design of the rewards function and system limitations, the charging action is biased towards the time of day when renewable energy output is maximum (midday). The charging decision by reinforcement learning improves the utilization of renewable energy by 2–4% compared to the random charging policy and the priority charging policy. This study contributes to the reduction in battery charging and discharging, electricity sold to the grid to create benefits and the reduction in carbon emissions.

Suggested Citation

  • Handong Li & Xuewu Dai & Stephen Goldrick & Richard Kotter & Nauman Aslam & Saleh Ali, 2024. "Reinforcement Learning for EV Fleet Smart Charging with On-Site Renewable Energy Sources," Energies, MDPI, vol. 17(21), pages 1-21, October.
  • Handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5442-:d:1511057
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/21/5442/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/17/21/5442/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hadi Suyono & Mir Toufikur Rahman & Hazlie Mokhlis & Mohamadariff Othman & Hazlee Azil Illias & Hasmaini Mohamad, 2019. "Optimal Scheduling of Plug-in Electric Vehicle Charging Including Time-of-Use Tariff to Minimize Cost and System Stress," Energies, MDPI, vol. 12(8), pages 1-21, April.
    2. Sadeghi-Barzani, Payam & Rajabi-Ghahnavieh, Abbas & Kazemi-Karegar, Hosein, 2014. "Optimal fast charging station placing and sizing," Applied Energy, Elsevier, vol. 125(C), pages 289-299.
    3. Neri, Manfredi & Guelpa, Elisa & Verda, Vittorio, 2022. "Design and connection optimization of a district cooling network: Mixed integer programming and heuristic approach," Applied Energy, Elsevier, vol. 306(PA).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Natascia Andrenacci & Roberto Ragona & Antonino Genovese, 2020. "Evaluation of the Instantaneous Power Demand of an Electric Charging Station in an Urban Scenario," Energies, MDPI, vol. 13(11), pages 1-19, May.
    2. Sun, Siyang & Yang, Qiang & Ma, Jin & Ferré, Adrià Junyent & Yan, Wenjun, 2020. "Hierarchical planning of PEV charging facilities and DGs under transportation-power network couplings," Renewable Energy, Elsevier, vol. 150(C), pages 356-369.
    3. Morro-Mello, Igoor & Padilha-Feltrin, Antonio & Melo, Joel D. & Heymann, Fabian, 2021. "Spatial connection cost minimization of EV fast charging stations in electric distribution networks using local search and graph theory," Energy, Elsevier, vol. 235(C).
    4. Neaimeh, Myriam & Salisbury, Shawn D. & Hill, Graeme A. & Blythe, Philip T. & Scoffield, Don R. & Francfort, James E., 2017. "Analysing the usage and evidencing the importance of fast chargers for the adoption of battery electric vehicles," Energy Policy, Elsevier, vol. 108(C), pages 474-486.
    5. Davidov, Sreten & Pantoš, Miloš, 2017. "Planning of electric vehicle infrastructure based on charging reliability and quality of service," Energy, Elsevier, vol. 118(C), pages 1156-1167.
    6. Se Hoon Baik & Young Gyu Jin & Yong Tae Yoon, 2018. "Determining Equipment Capacity of Electric Vehicle Charging Station Operator for Profit Maximization," Energies, MDPI, vol. 11(9), pages 1-15, September.
    7. Arlt, Marie-Louise & Astier, Nicolas, 2023. "Do retail businesses have efficient incentives to invest in public charging stations for electric vehicles?," Energy Economics, Elsevier, vol. 124(C).
    8. Kim, Hyunjung & Kim, Dae-Wook & Kim, Man-Keun, 2022. "Economics of charging infrastructure for electric vehicles in Korea," Energy Policy, Elsevier, vol. 164(C).
    9. Sultana, U. & Khairuddin, Azhar B. & Sultana, Beenish & Rasheed, Nadia & Qazi, Sajid Hussain & Malik, Nimra Riaz, 2018. "Placement and sizing of multiple distributed generation and battery swapping stations using grasshopper optimizer algorithm," Energy, Elsevier, vol. 165(PA), pages 408-421.
    10. Globisch, Joachim & Plötz, Patrick & Dütschke, Elisabeth & Wietschel, Martin, 2018. "Consumer evaluation of public charging infrastructure for electric vehicles," Working Papers "Sustainability and Innovation" S13/2018, Fraunhofer Institute for Systems and Innovation Research (ISI).
    11. Zhou, Guangyou & Zhu, Zhiwei & Luo, Sumei, 2022. "Location optimization of electric vehicle charging stations: Based on cost model and genetic algorithm," Energy, Elsevier, vol. 247(C).
    12. Milan Straka & Pasquale De Falco & Gabriella Ferruzzi & Daniela Proto & Gijs van der Poel & Shahab Khormali & v{L}ubov{s} Buzna, 2019. "Predicting popularity of EV charging infrastructure from GIS data," Papers 1910.02498, arXiv.org.
    13. Nimalsiri, Nanduni I. & Ratnam, Elizabeth L. & Mediwaththe, Chathurika P. & Smith, David B. & Halgamuge, Saman K., 2021. "Coordinated charging and discharging control of electric vehicles to manage supply voltages in distribution networks: Assessing the customer benefit," Applied Energy, Elsevier, vol. 291(C).
    14. Pietro Catrini & Tancredi Testasecca & Alessandro Buscemi & Antonio Piacentino, 2022. "Exergoeconomics as a Cost-Accounting Method in Thermal Grids with the Presence of Renewable Energy Producers," Sustainability, MDPI, vol. 14(7), pages 1-27, March.
    15. Das, H.S. & Rahman, M.M. & Li, S. & Tan, C.W., 2020. "Electric vehicles standards, charging infrastructure, and impact on grid integration: A technological review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 120(C).
    16. Wang, Yue & Shi, Jianmai & Wang, Rui & Liu, Zhong & Wang, Ling, 2018. "Siting and sizing of fast charging stations in highway network with budget constraint," Applied Energy, Elsevier, vol. 228(C), pages 1255-1271.
    17. Yıldız, Barış & Olcaytu, Evren & Şen, Ahmet, 2019. "The urban recharging infrastructure design problem with stochastic demands and capacitated charging stations," Transportation Research Part B: Methodological, Elsevier, vol. 119(C), pages 22-44.
    18. Arshdeep Singh & Shimi Sudha Letha, 2019. "Emerging energy sources for electric vehicle charging station," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 21(5), pages 2043-2082, October.
    19. Milad Akbari & Morris Brenna & Michela Longo, 2018. "Optimal Locating of Electric Vehicle Charging Stations by Application of Genetic Algorithm," Sustainability, MDPI, vol. 10(4), pages 1-14, April.
    20. Jiang, Huaiguang & Zhang, Yingchen & Chen, Yuche & Zhao, Changhong & Tan, Jin, 2018. "Power-traffic coordinated operation for bi-peak shaving and bi-ramp smoothing – A hierarchical data-driven approach," Applied Energy, Elsevier, vol. 229(C), pages 756-766.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5442-:d:1511057. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.