IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v185y2024ics1366554524000759.html
   My bibliography  Save this article

Recursive logit-based meta-inverse reinforcement learning for driver-preferred route planning

Author

Listed:
  • Zhang, Pujun
  • Lei, Dazhou
  • Liu, Shan
  • Jiang, Hai

Abstract

Driver-preferred route planning often evaluates the quality of a planned route based on how closely it is followed by the driver. Despite decades of research in this area, there still exist nonnegligible deviations from planned routes. Recently, with the prevalence of GPS data, Inverse Reinforcement Learning (IRL) has attracted much interest due to its ability to directly learn routing patterns from GPS trajectories. However, existing IRL methods are limited in that: (1) They rely on numerical approximations to calculate the expected state visitation frequencies (SVFs), which are inaccurate and time-consuming; and (2) They ignore the fact that the coverage of GPS trajectories is skewed toward popular road segments, causing difficulties in learning from sparsely covered ones. To overcome these challenges, we propose a recursive logit-based meta-IRL approach, where (1) We use the recursive logit model to capture drivers’ route choice behavior so that the expected SVFs can be analytically derived, which substantially reduces the computational efforts; and (2) We introduce meta-parameters and employ meta-learning techniques so that the learning on sparsely covered road segments can benefit from that on popular ones. When training our IRL model, we update the rewards of road segments with the expected SVFs by solving several systems of linear equations and update the meta-parameters through a two-level optimization structure to ensure its fast adaption and versatility. We validate our approach using real GPS data in Chengdu, China. Results show that our planned routes better match actual routes compared with state-of-the-art methods including the recursive logit model, Deep-IRL and Dij-IRL: the F1-Score increases by 4.17% with the introduction of the recursive logit model and further increases to 5.19% after meta-learning is employed. Moreover, we can reduce training time by over 95%.

Suggested Citation

  • Zhang, Pujun & Lei, Dazhou & Liu, Shan & Jiang, Hai, 2024. "Recursive logit-based meta-inverse reinforcement learning for driver-preferred route planning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 185(C).
  • Handle: RePEc:eee:transe:v:185:y:2024:i:c:s1366554524000759
    DOI: 10.1016/j.tre.2024.103485
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1366554524000759
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2024.103485?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yang, Lin & Kwan, Mei-Po & Pan, Xiaofang & Wan, Bo & Zhou, Shunping, 2017. "Scalable space-time trajectory cube for path-finding: A study using big taxi trajectory data," Transportation Research Part B: Methodological, Elsevier, vol. 101(C), pages 1-27.
    2. Fosgerau, Mogens & Frejinger, Emma & Karlstrom, Anders, 2013. "A link based network route choice model with unrestricted choice set," Transportation Research Part B: Methodological, Elsevier, vol. 56(C), pages 70-80.
    3. Mai, Tien & Fosgerau, Mogens & Frejinger, Emma, 2015. "A nested recursive logit model for route choice analysis," Transportation Research Part B: Methodological, Elsevier, vol. 75(C), pages 100-112.
    4. Nassir, Neema & Hickman, Mark & Ma, Zhen-Liang, 2019. "A strategy-based recursive path choice model for public transit smart card data," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 528-548.
    5. Dieter, Peter & Caron, Matthew & Schryen, Guido, 2023. "Integrating driver behavior into last-mile delivery routing: Combining machine learning and optimization in a hybrid decision support framework," European Journal of Operational Research, Elsevier, vol. 311(1), pages 283-300.
    6. Zhang, Pujun & Liu, Shan & Shi, Jia & Chen, Liying & Chen, Shuiping & Gao, Jiuchong & Jiang, Hai, 2023. "Route planning using divide-and-conquer: A GAT enhanced insertion transformer approach," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 176(C).
    7. Meyer de Freitas, Lucas & Becker, Henrik & Zimmermann, Maëlle & Axhausen, Kay W., 2019. "Modelling intermodal travel in Switzerland: A recursive logit approach," Transportation Research Part A: Policy and Practice, Elsevier, vol. 119(C), pages 200-213.
    8. Liu, Shan & Jiang, Hai, 2022. "Personalized route recommendation for ride-hailing with deep inverse reinforcement learning and real-time traffic conditions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    9. Tien Mai & Fabian Bastin & Emma Frejinger, 2018. "A decomposition method for estimating recursive logit based route choice models," EURO Journal on Transportation and Logistics, Springer;EURO - The Association of European Operational Research Societies, vol. 7(3), pages 253-275, September.
    10. Richard Bellman, 1957. "On a Dynamic Programming Approach to the Caterer Problem--I," Management Science, INFORMS, vol. 3(3), pages 270-278, April.
    11. Liu, Shan & Jiang, Hai & Chen, Shuiping & Ye, Jing & He, Renqing & Sun, Zhizhao, 2020. "Integrating Dijkstra’s algorithm into deep inverse reinforcement learning for food delivery route planning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 142(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cortés, Cristián E. & Donoso, Pedro & Gutiérrez, Leonel & Herl, Daniel & Muñoz, Diego, 2023. "A recursive stochastic transit equilibrium model estimated using passive data from Santiago, Chile," Transportation Research Part B: Methodological, Elsevier, vol. 174(C).
    2. Mai, Tien & Yu, Xinlian & Gao, Song & Frejinger, Emma, 2021. "Routing policy choice prediction in a stochastic network: Recursive model and solution algorithm," Transportation Research Part B: Methodological, Elsevier, vol. 151(C), pages 42-58.
    3. Liu, Shan & Jiang, Hai, 2022. "Personalized route recommendation for ride-hailing with deep inverse reinforcement learning and real-time traffic conditions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    4. Yuki Oyama, 2022. "Capturing positive network attributes during the estimation of recursive logit models: A prism-based approach," Papers 2204.01215, arXiv.org, revised Jan 2023.
    5. Tien Mai & The Viet Bui & Quoc Phong Nguyen & Tho V. Le, 2022. "Estimation of Recursive Route Choice Models with Incomplete Trip Observations," Papers 2204.12992, arXiv.org.
    6. Meyer de Freitas, Lucas & Becker, Henrik & Zimmermann, Maëlle & Axhausen, Kay W., 2019. "Modelling intermodal travel in Switzerland: A recursive logit approach," Transportation Research Part A: Policy and Practice, Elsevier, vol. 119(C), pages 200-213.
    7. Yao, Rui & Bekhor, Shlomo, 2022. "A variational autoencoder approach for choice set generation and implicit perception of alternatives in choice modeling," Transportation Research Part B: Methodological, Elsevier, vol. 158(C), pages 273-294.
    8. Hung Tran & Tien Mai, 2023. "Network-based Representations and Dynamic Discrete Choice Models for Multiple Discrete Choice Analysis," Papers 2306.04606, arXiv.org.
    9. van Oijen, Tim P. & Daamen, Winnie & Hoogendoorn, Serge P., 2020. "Estimation of a recursive link-based logit model and link flows in a sensor equipped network," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 262-281.
    10. Oyama, Yuki & Hato, Eiji, 2019. "Prism-based path set restriction for solving Markovian traffic assignment problem," Transportation Research Part B: Methodological, Elsevier, vol. 122(C), pages 528-546.
    11. Mai, Tien & Bui, The Viet & Nguyen, Quoc Phong & Le, Tho V., 2023. "Estimation of recursive route choice models with incomplete trip observations," Transportation Research Part B: Methodological, Elsevier, vol. 173(C), pages 313-331.
    12. Song, Yuchen & Li, Dawei & Liu, Dongjie & Cao, Qi & Chen, Junlan & Ren, Gang & Tang, Xiaoyong, 2022. "Modeling activity-travel behavior under a dynamic discrete choice framework with unobserved heterogeneity," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 167(C).
    13. Tran, Hung & Mai, Tien, 2024. "Network-based representations and dynamic discrete choice models for multiple discrete choice analysis," Transportation Research Part B: Methodological, Elsevier, vol. 184(C).
    14. Liu, Shan & Zhang, Ya & Wang, Zhengli & Gu, Shiyi, 2023. "AdaBoost-Bagging deep inverse reinforcement learning for autonomous taxi cruising route and speed planning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 177(C).
    15. Mogens Fosgerau & Mads Paulsen & Thomas Kj{ae}r Rasmussen, 2021. "A perturbed utility route choice model," Papers 2103.13784, arXiv.org, revised Sep 2021.
    16. Keerthana Sivamayil & Elakkiya Rajasekar & Belqasem Aljafari & Srete Nikolovski & Subramaniyaswamy Vairavasundaram & Indragandhi Vairavasundaram, 2023. "A Systematic Study on Reinforcement Learning Based Applications," Energies, MDPI, vol. 16(3), pages 1-23, February.
    17. Liu, Shan & Jiang, Hai & Chen, Shuiping & Ye, Jing & He, Renqing & Sun, Zhizhao, 2020. "Integrating Dijkstra’s algorithm into deep inverse reinforcement learning for food delivery route planning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 142(C).
    18. Evanthia Kazagli & Michel Bierlaire & Matthieu de Lapparent, 2020. "Operational route choice methodologies for practical applications," Transportation, Springer, vol. 47(1), pages 43-74, February.
    19. Kazagli, Evanthia & Bierlaire, Michel & Flötteröd, Gunnar, 2016. "Revisiting the route choice problem: A modeling framework based on mental representations," Journal of choice modelling, Elsevier, vol. 19(C), pages 1-23.
    20. Nassir, Neema & Hickman, Mark & Ma, Zhen-Liang, 2019. "A strategy-based recursive path choice model for public transit smart card data," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 528-548.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:185:y:2024:i:c:s1366554524000759. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.