IDEAS home Printed from https://ideas.repec.org/a/inm/ortrsc/v52y2018i3p691-706.html
   My bibliography  Save this article

Dynamic Orienteering on a Network of Queues

Author

Listed:
  • Shu Zhang

    (School of Economics and Business Administration, Chongqing University, 400044 Chongqing, China)

  • Jeffrey W. Ohlmann

    (Department of Management Sciences, Tippie College of Business, University of Iowa, Iowa City, Iowa 52242)

  • Barrett W. Thomas

    (Department of Management Sciences, Tippie College of Business, University of Iowa, Iowa City, Iowa 52242)

Abstract

We introduce a stochastic orienteering problem on a network of queues in which the traveler must arrive and enter service at locations within the respective time windows to collect rewards, but the traveler may experience stochastic wait time at each location before service can begin. To maximize the expected rewards collected, the traveler must determine which locations to visit and how long to wait in queues at each location. We formally model the problem as a Markov decision process with the objective of maximizing the expected collected rewards. We investigate the existence of optimal control limits and examine conditions under which certain actions cannot be optimal. To solve the problem, we propose an approximate dynamic programming approach based on rollout algorithms. The method introduces a two-stage heuristic estimation that we refer to as compound rollout. In the first stage, the algorithm decides whether to stay at the current location or go to another location. If departing the current location, it chooses the next location in the second stage. We demonstrate the value of our modeling and solution approaches by comparing the dynamic policies to a-priori-route solutions with recourse actions.

Suggested Citation

  • Shu Zhang & Jeffrey W. Ohlmann & Barrett W. Thomas, 2018. "Dynamic Orienteering on a Network of Queues," Transportation Science, INFORMS, vol. 52(3), pages 691-706, June.
    • Handle: RePEc:inm:ortrsc:v:52:y:2018:i:3:p:691-706
    DOI: 10.1287/trsc.2017.0761
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/trsc.2017.0761
    Download Restriction: no
    File URL: https://libkey.io/10.1287/trsc.2017.0761?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mathias A. Klapp & Alan L. Erera & Alejandro Toriello, 2018. "The One-Dimensional Dynamic Dispatch Waves Problem," Transportation Science, INFORMS, vol. 52(2), pages 402-415, March.
    2. Zhang, Shu & Ohlmann, Jeffrey W. & Thomas, Barrett W., 2014. "A priori orienteering with time windows and stochastic wait times at customers," European Journal of Operational Research, Elsevier, vol. 239(1), pages 70-79.
    3. Uri Yechiali, 1971. "On Optimal Balking Rules and Toll Charges in the GI / M /1 Queuing Process," Operations Research, INFORMS, vol. 19(2), pages 349-370, April.
    4. Nicola Secomandi, 2001. "A Rollout Policy for the Vehicle Routing Problem with Stochastic Demands," Operations Research, INFORMS, vol. 49(5), pages 796-802, October.
    5. Alejandro Toriello & William B. Haskell & Michael Poremba, 2014. "A Dynamic Traveling Salesman Problem with Stochastic Arc Costs," Operations Research, INFORMS, vol. 62(5), pages 1107-1125, October.
    6. H Tang & E Miller-Hooks, 2005. "Algorithms for a stochastic selective travelling salesperson problem," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(4), pages 439-452, April.
    7. Novoa, Clara & Storer, Robert, 2009. "An approximate dynamic programming approach for the vehicle routing problem with stochastic demands," European Journal of Operational Research, Elsevier, vol. 196(2), pages 509-515, July.
    8. Ann Campbell & Michel Gendreau & Barrett Thomas, 2011. "The orienteering problem with stochastic travel and service times," Annals of Operations Research, Springer, vol. 186(1), pages 61-81, June.
    9. Harsha Honnappa & Rahul Jain, 2015. "Strategic Arrivals into Queueing Networks: The Network Concert Queueing Game," Operations Research, INFORMS, vol. 63(1), pages 247-259, February.
    10. Vansteenwegen, Pieter & Souffriau, Wouter & Oudheusden, Dirk Van, 2011. "The orienteering problem: A survey," European Journal of Operational Research, Elsevier, vol. 209(1), pages 1-10, February.
    11. Justin C. Goodson & Jeffrey W. Ohlmann & Barrett W. Thomas, 2013. "Rollout Policies for Dynamic Solutions to the Multivehicle Routing Problem with Stochastic Demand and Duration Limits," Operations Research, INFORMS, vol. 61(1), pages 138-154, February.
    12. Uri Yechiali, 1972. "Customers' Optimal Joining Rules for the GI/M/s Queue," Management Science, INFORMS, vol. 18(7), pages 434-443, March.
    13. Apostolos Burnetas, 2013. "Customer equilibrium and optimal strategies in Markovian queues in series," Annals of Operations Research, Springer, vol. 208(1), pages 515-529, September.
    14. Justin C. Goodson & Barrett W. Thomas & Jeffrey W. Ohlmann, 2016. "Restocking-Based Rollout Policies for the Vehicle Routing Problem with Stochastic Demand and Duration Limits," Transportation Science, INFORMS, vol. 50(2), pages 591-607, May.
    15. Marius M. Solomon, 1987. "Algorithms for the Vehicle Routing and Scheduling Problems with Time Window Constraints," Operations Research, INFORMS, vol. 35(2), pages 254-265, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. John E. Fontecha & Oscar O. Guaje & Daniel Duque & Raha Akhavan-Tabatabaei & Juan P. Rodríguez & Andrés L. Medaglia, 2020. "Combined maintenance and routing optimization for large-scale sewage cleaning," Annals of Operations Research, Springer, vol. 286(1), pages 441-474, March.
    2. Soeffker, Ninja & Ulmer, Marlin W. & Mattfeld, Dirk C., 2022. "Stochastic dynamic vehicle routing in the light of prescriptive analytics: A review," European Journal of Operational Research, Elsevier, vol. 298(3), pages 801-820.
    3. Marlin W. Ulmer, 2020. "Horizontal combinations of online and offline approximate dynamic programming for stochastic dynamic vehicle routing," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 28(1), pages 279-308, March.
    4. Zhang, Shu & Ohlmann, Jeffrey W. & Thomas, Barrett W., 2020. "Multi-period orienteering with uncertain adoption likelihood and waiting at customers," European Journal of Operational Research, Elsevier, vol. 282(1), pages 288-303.
    5. Qinxiao Yu & Chun Cheng & Ning Zhu, 2022. "Robust Team Orienteering Problem with Decreasing Profits," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3215-3233, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Shu & Ohlmann, Jeffrey W. & Thomas, Barrett W., 2014. "A priori orienteering with time windows and stochastic wait times at customers," European Journal of Operational Research, Elsevier, vol. 239(1), pages 70-79.
    2. Zhang, Jian & Woensel, Tom Van, 2023. "Dynamic vehicle routing with random requests: A literature review," International Journal of Production Economics, Elsevier, vol. 256(C).
    3. Soeffker, Ninja & Ulmer, Marlin W. & Mattfeld, Dirk C., 2022. "Stochastic dynamic vehicle routing in the light of prescriptive analytics: A review," European Journal of Operational Research, Elsevier, vol. 298(3), pages 801-820.
    4. Wadi Khalid Anuar & Lai Soon Lee & Hsin-Vonn Seow & Stefan Pickl, 2022. "A Multi-Depot Dynamic Vehicle Routing Problem with Stochastic Road Capacity: An MDP Model and Dynamic Policy for Post-Decision State Rollout Algorithm in Reinforcement Learning," Mathematics, MDPI, vol. 10(15), pages 1-70, July.
    5. Zhang, Shu & Ohlmann, Jeffrey W. & Thomas, Barrett W., 2020. "Multi-period orienteering with uncertain adoption likelihood and waiting at customers," European Journal of Operational Research, Elsevier, vol. 282(1), pages 288-303.
    6. Wadi Khalid Anuar & Lai Soon Lee & Hsin-Vonn Seow & Stefan Pickl, 2021. "A Multi-Depot Vehicle Routing Problem with Stochastic Road Capacity and Reduced Two-Stage Stochastic Integer Linear Programming Models for Rollout Algorithm," Mathematics, MDPI, vol. 9(13), pages 1-44, July.
    7. Justin C. Goodson & Barrett W. Thomas & Jeffrey W. Ohlmann, 2016. "Restocking-Based Rollout Policies for the Vehicle Routing Problem with Stochastic Demand and Duration Limits," Transportation Science, INFORMS, vol. 50(2), pages 591-607, May.
    8. Lei, Chao & Lin, Wei-Hua & Miao, Lixin, 2014. "A multicut L-shaped based algorithm to solve a stochastic programming model for the mobile facility routing and scheduling problem," European Journal of Operational Research, Elsevier, vol. 238(3), pages 699-710.
    9. Dolinskaya, Irina & Shi, Zhenyu (Edwin) & Smilowitz, Karen, 2018. "Adaptive orienteering problem with stochastic travel times," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 109(C), pages 1-19.
    10. Verbeeck, C. & Vansteenwegen, P. & Aghezzaf, E.-H., 2016. "Solving the stochastic time-dependent orienteering problem with time windows," European Journal of Operational Research, Elsevier, vol. 255(3), pages 699-718.
    11. Majid Salavati-Khoshghalb & Michel Gendreau & Ola Jabali & Walter Rei, 2019. "A Rule-Based Recourse for the Vehicle Routing Problem with Stochastic Demands," Transportation Science, INFORMS, vol. 53(5), pages 1334-1353, September.
    12. Gunawan, Aldy & Lau, Hoong Chuin & Vansteenwegen, Pieter, 2016. "Orienteering Problem: A survey of recent variants, solution approaches and applications," European Journal of Operational Research, Elsevier, vol. 255(2), pages 315-332.
    13. Lucas Agussurja & Shih-Fen Cheng & Hoong Chuin Lau, 2019. "A State Aggregation Approach for Stochastic Multiperiod Last-Mile Ride-Sharing Problems," Service Science, INFORMS, vol. 53(1), pages 148-166, February.
    14. Florio, Alexandre M. & Hartl, Richard F. & Minner, Stefan, 2020. "Optimal a priori tour and restocking policy for the single-vehicle routing problem with stochastic demands," European Journal of Operational Research, Elsevier, vol. 285(1), pages 172-182.
    15. Qinxiao Yu & Chun Cheng & Ning Zhu, 2022. "Robust Team Orienteering Problem with Decreasing Profits," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3215-3233, November.
    16. Bertazzi, Luca & Secomandi, Nicola, 2018. "Faster rollout search for the vehicle routing problem with stochastic demands and restocking," European Journal of Operational Research, Elsevier, vol. 270(2), pages 487-497.
    17. Mathias A. Klapp & Alan L. Erera & Alejandro Toriello, 2018. "The One-Dimensional Dynamic Dispatch Waves Problem," Transportation Science, INFORMS, vol. 52(2), pages 402-415, March.
    18. Jorge Oyola & Halvard Arntzen & David L. Woodruff, 2017. "The stochastic vehicle routing problem, a literature review, Part II: solution methods," EURO Journal on Transportation and Logistics, Springer;EURO - The Association of European Operational Research Societies, vol. 6(4), pages 349-388, December.
    19. Marlin W. Ulmer & Barrett W. Thomas, 2019. "Enough Waiting for the Cable Guy—Estimating Arrival Times for Service Vehicle Routing," Transportation Science, INFORMS, vol. 53(3), pages 897-916, May.
    20. Klapp, Mathias A. & Erera, Alan L. & Toriello, Alejandro, 2018. "The Dynamic Dispatch Waves Problem for same-day delivery," European Journal of Operational Research, Elsevier, vol. 271(2), pages 519-534.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ortrsc:v:52:y:2018:i:3:p:691-706. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.