IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i3p1884-d743788.html
   My bibliography  Save this article

A Modified Rainbow-Based Deep Reinforcement Learning Method for Optimal Scheduling of Charging Station

Author

Listed:
  • Ruisheng Wang

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Zhong Chen

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Qiang Xing

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China
    School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore)

  • Ziqi Zhang

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Tian Zhang

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

Abstract

To improve the operating efficiency and economic benefits, this article proposes a modified rainbow-based deep reinforcement learning (DRL) strategy to realize the charging station (CS) optimal scheduling. As the charging process is a real-time matching between electric vehicles ‘(EVs) charging demand and CS equipment resources, the CS charging scheduling problem is duly formulated as a finite Markov decision process (FMDP). Considering the multi-stakeholder interaction among EVs, CSs, and distribution networks (DNs), a comprehensive information perception model was constructed to extract the environmental state required by the agent. According to the random behavior characteristics of the EV charging arrival and departure times, the startup of the charging pile control module was regarded as the agent’s action space. To tackle this issue, the modified rainbow approach was utilized to develop a time-scale-based CS scheme to compensate for the resource requirements mismatch on the energy scale. Case studies were conducted within a CS integrated with the photovoltaic and energy storage system. The results reveal that the proposed method effectively reduces the CS operating cost and improves the new energy consumption.

Suggested Citation

  • Ruisheng Wang & Zhong Chen & Qiang Xing & Ziqi Zhang & Tian Zhang, 2022. "A Modified Rainbow-Based Deep Reinforcement Learning Method for Optimal Scheduling of Charging Station," Sustainability, MDPI, vol. 14(3), pages 1-14, February.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:3:p:1884-:d:743788
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/3/1884/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/3/1884/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Raja S, Charles & Kumar N M, Vijaya & J, Senthil kumar & Nesamalar J, Jeslin Drusila, 2021. "Enhancing system reliability by optimally integrating PHEV charging station and renewable distributed generators: A Bi-Level programming approach," Energy, Elsevier, vol. 229(C).
    2. Zhang, Jing & Yan, Jie & Liu, Yongqian & Zhang, Haoran & Lv, Guoliang, 2020. "Daily electric vehicle charging load profiles considering demographics of vehicle users," Applied Energy, Elsevier, vol. 274(C).
    3. Chao Luo & Yih-Fang Huang & Vijay Gupta, 2018. "Stochastic Dynamic Pricing for EV Charging Stations with Renewables Integration and Energy Storage," Papers 1801.02128, arXiv.org.
    4. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    5. Dong, Feng & Liu, Yajie, 2020. "Policy evolution and effect evaluation of new-energy vehicle industry in China," Resources Policy, Elsevier, vol. 67(C).
    6. Ki-Beom Lee & Mohamed A. Ahmed & Dong-Ki Kang & Young-Chon Kim, 2020. "Deep Reinforcement Learning Based Optimal Route and Charging Station Selection," Energies, MDPI, vol. 13(23), pages 1-22, November.
    7. Leon Fidele Nishimwe H. & Sung-Guk Yoon, 2021. "Combined Optimal Planning and Operation of a Fast EV-Charging Station Integrated with Solar PV and ESS," Energies, MDPI, vol. 14(11), pages 1-18, May.
    8. Das, H.S. & Rahman, M.M. & Li, S. & Tan, C.W., 2020. "Electric vehicles standards, charging infrastructure, and impact on grid integration: A technological review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 120(C).
    9. Kapustin, Nikita O. & Grushevenko, Dmitry A., 2020. "Long-term electric vehicles outlook and their potential impact on electric grid," Energy Policy, Elsevier, vol. 137(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qiu, Dawei & Wang, Yi & Hua, Weiqi & Strbac, Goran, 2023. "Reinforcement learning for electric vehicle applications in power systems:A critical review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    2. Fescioglu-Unver, Nilgun & Yıldız Aktaş, Melike, 2023. "Electric vehicle charging service operations: A review of machine learning applications for infrastructure planning, control, pricing and routing," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    3. Ekaterina V. Orlova, 2023. "Dynamic Regimes for Corporate Human Capital Development Used Reinforcement Learning Methods," Mathematics, MDPI, vol. 11(18), pages 1-22, September.
    4. Ahmed M. Abed & Ali AlArjani, 2022. "The Neural Network Classifier Works Efficiently on Searching in DQN Using the Autonomous Internet of Things Hybridized by the Metaheuristic Techniques to Reduce the EVs’ Service Scheduling Time," Energies, MDPI, vol. 15(19), pages 1-25, September.
    5. Yuemin Zheng & Jin Tao & Qinglin Sun & Hao Sun & Zengqiang Chen & Mingwei Sun, 2023. "Adaptive Active Disturbance Rejection Load Frequency Control for Power System with Renewable Energies Using the Lyapunov Reward-Based Twin Delayed Deep Deterministic Policy Gradient Algorithm," Sustainability, MDPI, vol. 15(19), pages 1-25, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leonardo Nogueira Fontoura da Silva & Marcelo Bruno Capeletti & Alzenira da Rosa Abaide & Luciano Lopes Pfitscher, 2024. "A Stochastic Methodology for EV Fast-Charging Load Curve Estimation Considering the Highway Traffic and User Behavior," Energies, MDPI, vol. 17(7), pages 1-27, April.
    2. Powell, Siobhan & Cezar, Gustavo Vianna & Rajagopal, Ram, 2022. "Scalable probabilistic estimates of electric vehicle charging given observed driver behavior," Applied Energy, Elsevier, vol. 309(C).
    3. Li, Xiaohui & Wang, Zhenpo & Zhang, Lei & Sun, Fengchun & Cui, Dingsong & Hecht, Christopher & Figgener, Jan & Sauer, Dirk Uwe, 2023. "Electric vehicle behavior modeling and applications in vehicle-grid integration: An overview," Energy, Elsevier, vol. 268(C).
    4. Bruno Knevitz Hammerschmitt & Clodomiro Unsihuay-Vila & Jordan Passinato Sausen & Marcelo Bruno Capeletti & Alexandre Rasi Aoki & Mateus Duarte Teixeira & Carlos Henrique Barriquello & Alzenira da Ros, 2024. "Adaptive Charging Simulation Model for Different Electric Vehicles and Mobility Patterns," Energies, MDPI, vol. 17(16), pages 1-21, August.
    5. Jian Chen & Fangyi Li & Ranran Yang & Dawei Ma, 2020. "Impacts of Increasing Private Charging Piles on Electric Vehicles’ Charging Profiles: A Case Study in Hefei City, China," Energies, MDPI, vol. 13(17), pages 1-17, August.
    6. Zheng, Yanchong & Wang, Yubin & Yang, Qiang, 2023. "Two-phase operation for coordinated charging of electric vehicles in a market environment: From electric vehicle aggregators’ perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 171(C).
    7. Sahoo, Debajani & Harichandan, Sidhartha & Kar, Sanjay Kumar & S, Sreejesh, 2022. "An empirical study on consumer motives and attitude towards adoption of electric vehicles in India: Policy implications for stakeholders," Energy Policy, Elsevier, vol. 165(C).
    8. Marco Raugei & Alessio Peluso & Enrica Leccisi & Vasilis Fthenakis, 2021. "Life-Cycle Carbon Emissions and Energy Implications of High Penetration of Photovoltaics and Electric Vehicles in California," Energies, MDPI, vol. 14(16), pages 1-19, August.
    9. Wu, Wei & Lin, Boqiang, 2021. "Benefits of electric vehicles integrating into power grid," Energy, Elsevier, vol. 224(C).
    10. Lawrence Fulton, 2020. "A Publicly Available Simulation of Battery Electric, Hybrid Electric, and Gas-Powered Vehicles," Energies, MDPI, vol. 13(10), pages 1-15, May.
    11. Ahmed M. Abed & Ali AlArjani, 2022. "The Neural Network Classifier Works Efficiently on Searching in DQN Using the Autonomous Internet of Things Hybridized by the Metaheuristic Techniques to Reduce the EVs’ Service Scheduling Time," Energies, MDPI, vol. 15(19), pages 1-25, September.
    12. Chen, Yufeng & Ni, Liangfu & Liu, Kelong, 2021. "Does China's new energy vehicle industry innovate efficiently? A three-stage dynamic network slacks-based measure approach," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    13. Lei Ding & Xuejuan Fang, 2022. "Spatial–temporal distribution of air-pollution-intensive industries and its social-economic driving mechanism in Zhejiang Province, China: a framework of spatial econometric analysis," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 24(2), pages 1681-1712, February.
    14. Kandpal, Bakul & Pareek, Parikshit & Verma, Ashu, 2022. "A robust day-ahead scheduling strategy for EV charging stations in unbalanced distribution grid," Energy, Elsevier, vol. 249(C).
    15. Md. Mosaraf Hossain Khan & Amran Hossain & Aasim Ullah & Molla Shahadat Hossain Lipu & S. M. Shahnewaz Siddiquee & M. Shafiul Alam & Taskin Jamal & Hafiz Ahmed, 2021. "Integration of Large-Scale Electric Vehicles into Utility Grid: An Efficient Approach for Impact Analysis and Power Quality Assessment," Sustainability, MDPI, vol. 13(19), pages 1-18, October.
    16. Boud Verbrugge & Mohammed Mahedi Hasan & Haaris Rasool & Thomas Geury & Mohamed El Baghdadi & Omar Hegazy, 2021. "Smart Integration of Electric Buses in Cities: A Technological Review," Sustainability, MDPI, vol. 13(21), pages 1-23, November.
    17. Tuğba Yeğin & Muhammad Ikram, 2022. "Analysis of Consumers’ Electric Vehicle Purchase Intentions: An Expansion of the Theory of Planned Behavior," Sustainability, MDPI, vol. 14(19), pages 1-27, September.
    18. Feng Dong & Yuling Pan, 2020. "Evolution of Renewable Energy in BRI Countries: A Combined Econometric and Decomposition Approach," IJERPH, MDPI, vol. 17(22), pages 1-18, November.
    19. Parlikar, Anupam & Schott, Maximilian & Godse, Ketaki & Kucevic, Daniel & Jossen, Andreas & Hesse, Holger, 2023. "High-power electric vehicle charging: Low-carbon grid integration pathways with stationary lithium-ion battery systems and renewable generation," Applied Energy, Elsevier, vol. 333(C).
    20. Jingru Li & Jinxiao Ji & Jian Zuo & Yi Tan, 2023. "Is Policy the Necessary or Sufficient Driving Force of Construction and Demolition Waste Recycling Industry Development? Experience from China," IJERPH, MDPI, vol. 20(6), pages 1-16, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:3:p:1884-:d:743788. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.