IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v11y2019i14p3900-d249320.html
   My bibliography  Save this article

Congestion Control in Charging Stations Allocation with Q-Learning

Author

Listed:
  • Li Zhang

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

  • Ke Gong

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

  • Maozeng Xu

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

Abstract

Navigation systems can help in allocating public charging stations to electric vehicles (EVs) with the aim of minimizing EVs’ charging time by integrating sufficient data. However, the existing systems only consider their travel time and transform the allocation as a routing problem. In this paper, we involve the queuing time in stations as one part of EVs’ charging time, and another part is the travel time on roads. Roads and stations are easily congested resources, and we constructed a joint-resource congestion game to describe the interaction between vehicles and resources. With a finite number of vehicles and resources, there exists a Nash equilibrium. To realize a self-adaptive allocation work, we applied the Q-learning algorithm on systems, defining sets of states and actions in our constructed environment. After being allocated one by one, vehicles concurrently requesting to be charged will be processed properly. We collected urban road network data from Chongqing city and conducted experiments. The results illustrate the proposed method can be used to solve the problem, and its convergence performance was better than the genetic algorithm. The road capacity and the number of EVs affected the initial of Q-value, and not the convergence trends.

Suggested Citation

  • Li Zhang & Ke Gong & Maozeng Xu, 2019. "Congestion Control in Charging Stations Allocation with Q-Learning," Sustainability, MDPI, vol. 11(14), pages 1-11, July.
    • Handle: RePEc:gam:jsusta:v:11:y:2019:i:14:p:3900-:d:249320
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/11/14/3900/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/11/14/3900/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vazifeh, Mohammad M. & Zhang, Hongmou & Santi, Paolo & Ratti, Carlo, 2019. "Optimizing the deployment of electric vehicle charging stations using pervasive mobility data," Transportation Research Part A: Policy and Practice, Elsevier, vol. 121(C), pages 75-91.
    2. Haoming Liu & Wenqian Yin & Xiaoling Yuan & Man Niu, 2018. "Reserving Charging Decision-Making Model and Route Plan for Electric Vehicles Considering Information of Traffic and Charging Station," Sustainability, MDPI, vol. 10(5), pages 1-20, April.
    3. Luo, Lizi & Gu, Wei & Wu, Zhi & Zhou, Suyang, 2019. "Joint planning of distributed generation and electric vehicle charging stations considering real-time charging navigation," Applied Energy, Elsevier, vol. 242(C), pages 1274-1284.
    4. Dan-Bi Bak & Jae-Seok Bak & Sung-Yul Kim, 2018. "Strategies for Implementing Public Service Electric Bus Lines by Charging Type in Daegu Metropolitan City, South Korea," Sustainability, MDPI, vol. 10(10), pages 1-16, September.
    5. Shaohua Cui & Hui Zhao & Huijie Wen & Cuiping Zhang, 2018. "Locating Multiple Size and Multiple Type of Charging Station for Battery Electricity Vehicles," Sustainability, MDPI, vol. 10(9), pages 1-20, September.
    6. Simona Bigerna & Silvia Micheli, 2018. "Attitudes Toward Electric Vehicles: The Case of Perugia Using a Fuzzy Set Analysis," Sustainability, MDPI, vol. 10(11), pages 1-14, November.
    7. Milad Akbari & Morris Brenna & Michela Longo, 2018. "Optimal Locating of Electric Vehicle Charging Stations by Application of Genetic Algorithm," Sustainability, MDPI, vol. 10(4), pages 1-14, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Junchi Ma & Yuan Zhang & Zongtao Duan & Lei Tang, 2023. "PROLIFIC: Deep Reinforcement Learning for Efficient EV Fleet Scheduling and Charging," Sustainability, MDPI, vol. 15(18), pages 1-22, September.
    2. Mevan Wijewardena & Michael J. Neely, 2023. "A Two-Player Resource-Sharing Game with Asymmetric Information," Games, MDPI, vol. 14(5), pages 1-27, September.
    3. Fescioglu-Unver, Nilgun & Yıldız Aktaş, Melike, 2023. "Electric vehicle charging service operations: A review of machine learning applications for infrastructure planning, control, pricing and routing," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    4. Kim, Hyunjung & Kim, Dae-Wook & Kim, Man-Keun, 2022. "Economics of charging infrastructure for electric vehicles in Korea," Energy Policy, Elsevier, vol. 164(C).
    5. Khalil Bachiri & Ali Yahyaouy & Hamid Gualous & Maria Malek & Younes Bennani & Philippe Makany & Nicoleta Rogovschi, 2023. "Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation," Energies, MDPI, vol. 16(16), pages 1-17, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emilia M. Szumska & Rafał S. Jurecki, 2021. "Parameters Influencing on Electric Vehicle Range," Energies, MDPI, vol. 14(16), pages 1-23, August.
    2. Mikołaj Schmidt & Paweł Zmuda-Trzebiatowski & Marcin Kiciński & Piotr Sawicki & Konrad Lasak, 2021. "Multiple-Criteria-Based Electric Vehicle Charging Infrastructure Design Problem," Energies, MDPI, vol. 14(11), pages 1-34, May.
    3. Jiwon Lee & Midam An & Yongku Kim & Jung-In Seo, 2021. "Optimal Allocation for Electric Vehicle Charging Stations," Energies, MDPI, vol. 14(18), pages 1-10, September.
    4. Jean-Michel Clairand & Paulo Guerra-Terán & Xavier Serrano-Guerrero & Mario González-Rodríguez & Guillermo Escrivá-Escrivá, 2019. "Electric Vehicles for Public Transportation in Power Systems: A Review of Methodologies," Energies, MDPI, vol. 12(16), pages 1-22, August.
    5. Raja S, Charles & Kumar N M, Vijaya & J, Senthil kumar & Nesamalar J, Jeslin Drusila, 2021. "Enhancing system reliability by optimally integrating PHEV charging station and renewable distributed generators: A Bi-Level programming approach," Energy, Elsevier, vol. 229(C).
    6. Xie, Shiwei & Hu, Zhijian & Wang, Jueying & Chen, Yuwei, 2020. "The optimal planning of smart multi-energy systems incorporating transportation, natural gas and active distribution networks," Applied Energy, Elsevier, vol. 269(C).
    7. Shaohua Cui & Hui Zhao & Cuiping Zhang, 2018. "Locating Charging Stations of Various Sizes with Different Numbers of Chargers for Battery Electric Vehicles," Energies, MDPI, vol. 11(11), pages 1-22, November.
    8. Woo, Hyeon & Son, Yongju & Cho, Jintae & Kim, Sung-Yul & Choi, Sungyun, 2023. "Optimal expansion planning of electric vehicle fast charging stations," Applied Energy, Elsevier, vol. 342(C).
    9. Zhou, Guangyou & Zhu, Zhiwei & Luo, Sumei, 2022. "Location optimization of electric vehicle charging stations: Based on cost model and genetic algorithm," Energy, Elsevier, vol. 247(C).
    10. Mariano Gallo & Mario Marinelli, 2020. "Sustainable Mobility: A Review of Possible Actions and Policies," Sustainability, MDPI, vol. 12(18), pages 1-39, September.
    11. Mohd Bilal & Ibrahim Alsaidan & Muhannad Alaraj & Fahad M. Almasoudi & Mohammad Rizwan, 2022. "Techno-Economic and Environmental Analysis of Grid-Connected Electric Vehicle Charging Station Using AI-Based Algorithm," Mathematics, MDPI, vol. 10(6), pages 1-40, March.
    12. Feifeng Zheng & Zhaojie Wang & Ming Liu, 2022. "Overnight charging scheduling of battery electric buses with uncertain charging time," Operational Research, Springer, vol. 22(5), pages 4865-4903, November.
    13. Wenz, Klaus-Peter & Serrano-Guerrero, Xavier & Barragán-Escandón, Antonio & González, L.G. & Clairand, Jean-Michel, 2021. "Route prioritization of urban public transportation from conventional to electric buses: A new methodology and a study of case in an intermediate city of Ecuador," Renewable and Sustainable Energy Reviews, Elsevier, vol. 148(C).
    14. Armando Cartenì & Ilaria Henke & Clorinda Molitierno & Luigi Di Francesco, 2020. "Strong Sustainability in Public Transport Policies: An e-Mobility Bus Fleet Application in Sorrento Peninsula (Italy)," Sustainability, MDPI, vol. 12(17), pages 1-19, August.
    15. Elena Higueras-Castillo & Sebastian Molinillo & J. Andres Coca-Stefaniak & Francisco Liébana-Cabanillas, 2019. "Perceived Value and Customer Adoption of Electric and Hybrid Vehicles," Sustainability, MDPI, vol. 11(18), pages 1-15, September.
    16. Anders F. Jensen & Thomas K. Rasmussen & Carlo G. Prato, 2020. "A Route Choice Model for Capturing Driver Preferences When Driving Electric and Conventional Vehicles," Sustainability, MDPI, vol. 12(3), pages 1-18, February.
    17. Wei Wei & Ming Cao & Qianling Jiang & Sheng-Jung Ou & Hong Zou, 2020. "What Influences Chinese Consumers’ Adoption of Battery Electric Vehicles? A Preliminary Study Based on Factor Analysis," Energies, MDPI, vol. 13(5), pages 1-15, February.
    18. Lin, Haiyang & Bian, Caiyun & Wang, Yu & Li, Hailong & Sun, Qie & Wallin, Fredrik, 2022. "Optimal planning of intra-city public charging stations," Energy, Elsevier, vol. 238(PC).
    19. Park, Hyunwoo & Lee, Chungmok, 2024. "An exact algorithm for maximum electric vehicle flow coverage problem with heterogeneous chargers, nonlinear charging time and route deviations," European Journal of Operational Research, Elsevier, vol. 315(3), pages 926-951.
    20. Scorrano, Mariangela & Danielis, Romeo & Giansoldati, Marco, 2020. "Dissecting the total cost of ownership of fully electric cars in Italy: The impact of annual distance travelled, home charging and urban driving," Research in Transportation Economics, Elsevier, vol. 80(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:11:y:2019:i:14:p:3900-:d:249320. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.