IDEAS home Printed from https://ideas.repec.org/a/eee/proeco/v240y2021ics0925527321002279.html
   My bibliography  Save this article

Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system

Author

Listed:
  • Qin, Wei
  • Sun, Yan-Ning
  • Zhuang, Zi-Long
  • Lu, Zhi-Yao
  • Zhou, Yao-Ming

Abstract

The task assignment for vehicles plays an important role in urban transportation system, which is the key to cost reduction and efficiency improvement. The development of information technology and the emergence of “sharing economy” create a more convenient transportation mode, but also bring a greater challenge to efficient operation of urban transportation system. On the one hand, considering the complex and dynamic environment of urban transportation, an efficient method for assigning transportation tasks to idle vehicles is desired. On the other hand, to meet the users' expectations on immediate response of vehicle, the task assignment problem with dynamic arrival remains to be resolved. In this study, we propose a dynamic task assignment method for vehicles in urban transportation system based on the multi-agent reinforcement learning (RL). The transportation task assignment problem is transformed into a stochastic game process from vehicles’ perspective, and then an extended actor-critic (AC) algorithm is employed to obtain the optimal strategy. Based on the proposed method, vehicles can independently make decisions in real time, thus eliminating a lot of communication cost. Compared with the methods based on first-come-first-service (FCFS) rule and classic contract net algorithm (CNA), the results show that the proposed method can obtain higher acceptance rate and profit rate in the service cycle.

Suggested Citation

  • Qin, Wei & Sun, Yan-Ning & Zhuang, Zi-Long & Lu, Zhi-Yao & Zhou, Yao-Ming, 2021. "Multi-agent reinforcement learning-based dynamic task assignment for vehicles in urban transportation system," International Journal of Production Economics, Elsevier, vol. 240(C).
    • Handle: RePEc:eee:proeco:v:240:y:2021:i:c:s0925527321002279
    DOI: 10.1016/j.ijpe.2021.108251
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0925527321002279
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijpe.2021.108251?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gabrel, Virginie & Vanderpooten, Daniel, 2002. "Enumeration and interactive selection of efficient paths in a multiple criteria graph for scheduling an earth observing satellite," European Journal of Operational Research, Elsevier, vol. 139(3), pages 533-542, June.
    2. Lu Zhen & Shucheng Yu & Shuaian Wang & Zhuo Sun, 2019. "Scheduling quay cranes and yard trucks for unloading operations in container ports," Annals of Operations Research, Springer, vol. 273(1), pages 455-478, February.
    3. Morin, Michael & Gaudreault, Jonathan & Brotherton, Edith & Paradis, Frédérik & Rolland, Amélie & Wery, Jean & Laviolette, François, 2020. "Machine learning-based models of sawmills for better wood allocation planning," International Journal of Production Economics, Elsevier, vol. 222(C).
    4. Anton J. Kleywegt & Jason D. Papastavrou, 1998. "The Dynamic and Stochastic Knapsack Problem," Operations Research, INFORMS, vol. 46(1), pages 17-35, February.
    5. Ray Y. Zhong & Chen Xu & Chao Chen & George Q. Huang, 2017. "Big Data Analytics for Physical Internet-based intelligent manufacturing shop floors," International Journal of Production Research, Taylor & Francis Journals, vol. 55(9), pages 2610-2621, May.
    6. Russell, Robert A., 2017. "Mathematical programming heuristics for the production routing problem," International Journal of Production Economics, Elsevier, vol. 193(C), pages 40-49.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jung, Seung Hwan & Yang, Yunsi, 2023. "On the value of operational flexibility in the trailer shipment and assignment problem: Data-driven approaches and reinforcement learning," International Journal of Production Economics, Elsevier, vol. 264(C).
    2. Yongtao Peng & Bohai Chen & Eleonora Veglianti, 2022. "Platform Service Supply Chain Network Equilibrium Model with Data Empowerment," Sustainability, MDPI, vol. 14(9), pages 1-21, April.
    3. Sarkar, Mitali & Dey, Bikash Koli & Ganguly, Baishakhi & Saxena, Neha & Yadav, Dharmendra & Sarkar, Biswajit, 2023. "The impact of information sharing and bullwhip effects on improving consumer services in dual-channel retailing," Journal of Retailing and Consumer Services, Elsevier, vol. 73(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adrian Lee & Sheldon Jacobson, 2011. "Sequential stochastic assignment under uncertainty: estimation and convergence," Statistical Inference for Stochastic Processes, Springer, vol. 14(1), pages 21-46, February.
    2. Feng, Youyi & Xiao, Baichun, 2006. "A continuous-time seat control model for single-leg flights with no-shows and optimal overbooking upper bound," European Journal of Operational Research, Elsevier, vol. 174(2), pages 1298-1316, October.
    3. Dmitry Ivanov, 2022. "Viable supply chain model: integrating agility, resilience and sustainability perspectives—lessons from and thinking beyond the COVID-19 pandemic," Annals of Operations Research, Springer, vol. 319(1), pages 1411-1431, December.
    4. Alexander G. Nikolaev & Sheldon H. Jacobson, 2010. "Technical Note ---Stochastic Sequential Decision-Making with a Random Number of Jobs," Operations Research, INFORMS, vol. 58(4-part-1), pages 1023-1027, August.
    5. Xuanjing Fang & Yanan Du & Yuzhuo Qiu, 2017. "Reducing Carbon Emissions in a Closed-Loop Production Routing Problem with Simultaneous Pickups and Deliveries under Carbon Cap-and-Trade," Sustainability, MDPI, vol. 9(12), pages 1-15, November.
    6. Rigo, Cezar Antônio & Seman, Laio Oriel & Camponogara, Eduardo & Morsch Filho, Edemar & Bezerra, Eduardo Augusto & Munari, Pedro, 2022. "A branch-and-price algorithm for nanosatellite task scheduling to improve mission quality-of-service," European Journal of Operational Research, Elsevier, vol. 303(1), pages 168-183.
    7. Jeffrey I. McGill & Garrett J. van Ryzin, 1999. "Revenue Management: Research Overview and Prospects," Transportation Science, INFORMS, vol. 33(2), pages 233-256, May.
    8. Wei Zhang & Sriram Dasu & Reza Ahmadi, 2017. "Higher Prices for Larger Quantities? Nonmonotonic Price–Quantity Relations in B2B Markets," Management Science, INFORMS, vol. 63(7), pages 2108-2126, July.
    9. Diclehan Tezcaner & Murat Köksalan, 2011. "An Interactive Algorithm for Multi-objective Route Planning," Journal of Optimization Theory and Applications, Springer, vol. 150(2), pages 379-394, August.
    10. Yadi Zhao & Lei Yan & Jian Wu & Ximing Song, 2023. "Design and Implementation of a Digital Twin System for Log Rotary Cutting Optimization," Future Internet, MDPI, vol. 16(1), pages 1-14, December.
    11. Diego Muñoz-Carpintero & Doris Sáez & Cristián E. Cortés & Alfredo Núñez, 2015. "A Methodology Based on Evolutionary Algorithms to Solve a Dynamic Pickup and Delivery Problem Under a Hybrid Predictive Control Approach," Transportation Science, INFORMS, vol. 49(2), pages 239-253, May.
    12. Masoud Zafarzadeh & Magnus Wiktorsson & Jannicke Baalsrud Hauge, 2021. "A Systematic Review on Technologies for Data-Driven Production Logistics: Their Role from a Holistic and Value Creation Perspective," Logistics, MDPI, vol. 5(2), pages 1-32, April.
    13. Liu, Weihua & Long, Shangsong & Wei, Shuang, 2022. "Correlation mechanism between smart technology and smart supply chain innovation performance: A multi-case study from China's companies with Physical Internet," International Journal of Production Economics, Elsevier, vol. 245(C).
    14. Keumseok Kang & J. George Shanthikumar & Kemal Altinkemer, 2016. "Postponable Acceptance and Assignment: A Stochastic Dynamic Programming Approach," Manufacturing & Service Operations Management, INFORMS, vol. 18(4), pages 493-508, October.
    15. Dalila B. M. M. Fontes & S. Mahdi Homayouni, 2023. "A bi-objective multi-population biased random key genetic algorithm for joint scheduling quay cranes and speed adjustable vehicles in container terminals," Flexible Services and Manufacturing Journal, Springer, vol. 35(1), pages 241-268, March.
    16. Sundarakani, Balan & Ajaykumar, Aneesh & Gunasekaran, Angappa, 2021. "Big data driven supply chain design and applications for blockchain: An action research using case study approach," Omega, Elsevier, vol. 102(C).
    17. Richard Van Slyke & Yi Young, 2000. "Finite Horizon Stochastic Knapsacks with Applications to Yield Management," Operations Research, INFORMS, vol. 48(1), pages 155-172, February.
    18. Claudio Vitari & Elisabetta Raguseo, 2019. "Big data analytics business value and firm performance: Linking with environmental context," Post-Print hal-02293765, HAL.
    19. Kalyan Talluri & Garrett van Ryzin, 2000. "Revenue management under general discrete choice model of consumer behavior," Economics Working Papers 533, Department of Economics and Business, Universitat Pompeu Fabra, revised Oct 2001.
    20. Shenle Pan, 2019. "Opportunities of Product-Service System in Physical Internet," Post-Print hal-02155622, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:proeco:v:240:y:2021:i:c:s0925527321002279. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijpe .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.