IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v179y2023ics1366554523002831.html
   My bibliography  Save this article

Towards efficient airline disruption recovery with reinforcement learning

Author

Listed:
  • Ding, Yida
  • Wandelt, Sebastian
  • Wu, Guohua
  • Xu, Yifan
  • Sun, Xiaoqian

Abstract

Disruptions to airline schedules precipitate flight delays/cancellations and significant losses for airline operations. The goal of the integrated airline recovery problem is to develop an operational tool that provides the airline with an instant and cost-effective solution concerning aircraft, crew members and passengers in face of the emerging disruptions. In this paper, we formulate a decision recommendation framework which incorporates various recovery decisions including aircraft and crew rerouting, passenger reaccommodation, departure holding, flight cancellation and cruise speed control. Given the computational hardness of solving the mixed-integer nonlinear programming (MINP) model by the commercial solver (e.g., CPLEX), we establish a novel solution framework by incorporating Deep Reinforcement Learning (DRL) to the Variable Neighborhood Search (VNS) algorithm with well-designed neighborhood structures and state evaluator. We utilize Proximal Policy Optimization (PPO) to train the stochastic policy exploited to select neighborhood operations given the current state throughout the Markov Decision Process (MDP). Experimental results show that the objective value generated by our approach is within a 1.5% gap with respect to the optimal/close-to-optimal objective of the CPLEX solver for the small-scale instances, with significant improvement regarding runtime. The pre-trained DRL agent can leverage features/weights obtained from the training process to accelerate the arrival of objective convergence and further improve solution quality, which exhibits the potential of achieving Transfer Learning (TL). Given the inherent intractability of the problem on practical size instances, we propose a method to control the size of the DRL agent’s action space to allow for efficient training process. We believe our study contributes to the efforts of airlines in seeking efficient and cost-effective recovery solutions.

Suggested Citation

  • Ding, Yida & Wandelt, Sebastian & Wu, Guohua & Xu, Yifan & Sun, Xiaoqian, 2023. "Towards efficient airline disruption recovery with reinforcement learning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 179(C).
  • Handle: RePEc:eee:transe:v:179:y:2023:i:c:s1366554523002831
    DOI: 10.1016/j.tre.2023.103295
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1366554523002831
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2023.103295?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Benjamin G. Thengvall & Jonathan F. Bard & Gang Yu, 2003. "A Bundle Algorithm Approach for the Aircraft Schedule Recovery Problem During Hub Closures," Transportation Science, INFORMS, vol. 37(4), pages 392-407, November.
    2. Basso, Rafael & Kulcsár, Balázs & Sanchez-Diaz, Ivan & Qu, Xiaobo, 2022. "Dynamic stochastic electric vehicle routing with safe reinforcement learning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 157(C).
    3. Mohamed Haouari & Farah Zeghal Mansour & Hanif D. Sherali, 2019. "A New Compact Formulation for the Daily Crew Pairing Problem," Transportation Science, INFORMS, vol. 53(3), pages 811-828, May.
    4. James F. Campbell & Morton E. O'Kelly, 2012. "Twenty-Five Years of Hub Location Research," Transportation Science, INFORMS, vol. 46(2), pages 153-169, May.
    5. Wen, Xin & Chung, Sai-Ho & Ji, Ping & Sheu, Jiuh-Biing, 2022. "Individual scheduling approach for multi-class airline cabin crew with manpower requirement heterogeneity," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 163(C).
    6. Yan, Yimo & Chow, Andy H.F. & Ho, Chin Pang & Kuo, Yong-Hong & Wu, Qihao & Ying, Chengshuo, 2022. "Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 162(C).
    7. O'Kelly, M. E. & Bryan, D. L., 1998. "Hub location with flow economies of scale," Transportation Research Part B: Methodological, Elsevier, vol. 32(8), pages 605-616, November.
    8. Khan, Waqar Ahmed & Ma, Hoi-Lam & Ouyang, Xu & Mo, Daniel Y., 2021. "Prediction of aircraft trajectory and the associated fuel consumption using covariance bidirectional extreme learning machines," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 145(C).
    9. Šemrov, D. & Marsetič, R. & Žura, M. & Todorovski, L. & Srdic, A., 2016. "Reinforcement learning approach for train rescheduling on a single-track railway," Transportation Research Part B: Methodological, Elsevier, vol. 86(C), pages 250-267.
    10. Wen, Xin & Sun, Xuting & Sun, Yige & Yue, Xiaohang, 2021. "Airline crew scheduling: Models and algorithms," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 149(C).
    11. Xiong, Jing & Hansen, Mark, 2013. "Modelling airline flight cancellation decisions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 56(C), pages 64-80.
    12. Hu, Yuzhen & Song, Yan & Zhao, Kang & Xu, Baoguang, 2016. "Integrated recovery of aircraft and passengers after airline operation disruption based on a GRASP algorithm," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 87(C), pages 97-112.
    13. Wen, Xin & Ma, Hoi-Lam & Chung, Sai-Ho & Khan, Waqar Ahmed, 2020. "Robust airline crew scheduling with flight flying time variability," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 144(C).
    14. Jane Lee & Lavanya Marla & Alexandre Jacquillat, 2020. "Dynamic Disruption Management in Airline Networks Under Airport Operating Uncertainty," Transportation Science, INFORMS, vol. 54(4), pages 973-997, July.
    15. Liang, Zhe & Xiao, Fan & Qian, Xiongwen & Zhou, Lei & Jin, Xianfei & Lu, Xuehua & Karichery, Sureshan, 2018. "A column generation-based heuristic for aircraft recovery problem with airport capacity constraints and maintenance flexibility," Transportation Research Part B: Methodological, Elsevier, vol. 113(C), pages 70-90.
    16. Filom, Siyavash & Amiri, Amir M. & Razavi, Saiedeh, 2022. "Applications of machine learning methods in port operations – A systematic literature review," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 161(C).
    17. Dennis, Nigel, 2007. "End of the free lunch? The responses of traditional European airlines to the low-cost carrier threat," Journal of Air Transport Management, Elsevier, vol. 13(5), pages 311-321.
    18. Xu, Yifan & Wandelt, Sebastian & Sun, Xiaoqian, 2021. "Airline integrated robust scheduling with a variable neighborhood search based heuristic," Transportation Research Part B: Methodological, Elsevier, vol. 149(C), pages 181-203.
    19. Cynthia Barnhart & Natashia L. Boland & Lloyd W. Clarke & Ellis L. Johnson & George L. Nemhauser & Rajesh G. Shenoi, 1998. "Flight String Models for Aircraft Fleeting and Routing," Transportation Science, INFORMS, vol. 32(3), pages 208-220, August.
    20. Lavanya Marla & Bo Vaaben & Cynthia Barnhart, 2017. "Integrated Disruption Management and Flight Planning to Trade Off Delays and Fuel Burn," Transportation Science, INFORMS, vol. 51(1), pages 88-111, February.
    21. Cynthia Barnhart & Douglas Fearing & Vikrant Vaze, 2014. "Modeling Passenger Travel and Delays in the National Air Transportation System," Operations Research, INFORMS, vol. 62(3), pages 580-601, June.
    22. Herrema, Floris & Curran, Ricky & Hartjes, Sander & Ellejmi, Mohamed & Bancroft, Steven & Schultz, Michael, 2019. "A machine learning model to predict runway exit at Vienna airport," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 131(C), pages 329-342.
    23. Alcaraz, Juan J. & Losilla, Fernando & Caballero-Arnaldos, Luis, 2022. "Online model-based reinforcement learning for decision-making in long distance routes," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    24. Kohl, Niklas & Larsen, Allan & Larsen, Jesper & Ross, Alex & Tiourine, Sergey, 2007. "Airline disruption management—Perspectives, experiences and outlook," Journal of Air Transport Management, Elsevier, vol. 13(3), pages 149-162.
    25. Zuidberg, Joost, 2014. "Identifying airline cost economies: An econometric analysis of the factors affecting aircraft operating costs," Journal of Air Transport Management, Elsevier, vol. 40(C), pages 86-95.
    26. Jon D. Petersen & Gustaf Sölveling & John-Paul Clarke & Ellis L. Johnson & Sergey Shebalov, 2012. "An Optimization Approach to Airline Integrated Recovery," Transportation Science, INFORMS, vol. 46(4), pages 482-500, November.
    27. Bongiovanni, Claudia & Kaspi, Mor & Cordeau, Jean-François & Geroliminis, Nikolas, 2022. "A machine learning-driven two-phase metaheuristic for autonomous ridesharing operations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 165(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Winkelmann, Jonas & Spinler, Stefan & Neukirchen, Thomas, 2024. "Green transport fleet renewal using approximate dynamic programming: A case study in German heavy-duty road transportation," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 186(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schrotenboer, Albert H. & Wenneker, Rob & Ursavas, Evrim & Zhu, Stuart X., 2023. "Reliable reserve-crew scheduling for airlines," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 178(C).
    2. Huang, Lei & Xiao, Fan & Zhou, Jing & Duan, Zhenya & Zhang, Hua & Liang, Zhe, 2023. "A machine learning based column-and-row generation approach for integrated air cargo recovery problem," Transportation Research Part B: Methodological, Elsevier, vol. 178(C).
    3. Wen, Xin & Sun, Xuting & Ma, Hoi-Lam & Sun, Yige, 2022. "A column generation approach for operational flight scheduling and aircraft maintenance routing," Journal of Air Transport Management, Elsevier, vol. 105(C).
    4. Naz Yeti̇moğlu, Yücel & Selim Aktürk, M., 2021. "Aircraft and passenger recovery during an aircraft’s unexpected unavailability," Journal of Air Transport Management, Elsevier, vol. 91(C).
    5. Huang, Zhouchun & Luo, Xiaodong & Jin, Xianfei & Karichery, Sureshan, 2022. "An iterative cost-driven copy generation approach for aircraft recovery problem," European Journal of Operational Research, Elsevier, vol. 301(1), pages 334-348.
    6. Nianyi Wang & Huiling Wang & Shan Pei & Boyu Zhang, 2023. "A Data-Driven Heuristic Method for Irregular Flight Recovery," Mathematics, MDPI, vol. 11(11), pages 1-22, June.
    7. Ma, Hoi-Lam & Sun, Yige & Chung, Sai-Ho & Chan, Hing Kai, 2022. "Tackling uncertainties in aircraft maintenance routing: A review of emerging technologies," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    8. Birolini, Sebastian & Jacquillat, Alexandre & Cattaneo, Mattia & Antunes, António Pais, 2021. "Airline Network Planning: Mixed-integer non-convex optimization with demand–supply interactions," Transportation Research Part B: Methodological, Elsevier, vol. 154(C), pages 100-124.
    9. Pedro Jose Gudiel Pineda & Chao-Che Hsu & James J. H. Liou & Huai-Wei Lo, 2018. "A Hybrid Model for Aircraft Type Determination Following Flight Cancellation," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(04), pages 1147-1172, July.
    10. Li, Max Z. & Ryerson, Megan S., 2019. "Reviewing the DATAS of aviation research data: Diversity, availability, tractability, applicability, and sources," Journal of Air Transport Management, Elsevier, vol. 75(C), pages 111-130.
    11. Jane Lee & Lavanya Marla & Alexandre Jacquillat, 2020. "Dynamic Disruption Management in Airline Networks Under Airport Operating Uncertainty," Transportation Science, INFORMS, vol. 54(4), pages 973-997, July.
    12. Xiao, Fan & Guo, Siqi & Huang, Lin & Huang, Lei & Liang, Zhe, 2022. "Integrated aircraft tail assignment and cargo routing problem with through cargo consideration," Transportation Research Part B: Methodological, Elsevier, vol. 162(C), pages 328-351.
    13. Wen, Xin & Chung, Sai-Ho & Ji, Ping & Sheu, Jiuh-Biing, 2022. "Individual scheduling approach for multi-class airline cabin crew with manpower requirement heterogeneity," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 163(C).
    14. Ding, Chengjin & Chen, Xinyuan & Wu, Weiwei & Wei, Wenbin & Xin, Zelin, 2023. "Game-theoretic analysis of the impact of crew overnight hotel cost on airlines’ fleet assignment and crew pairing," Journal of Air Transport Management, Elsevier, vol. 113(C).
    15. Evler, Jan & Asadi, Ehsan & Preis, Henning & Fricke, Hartmut, 2021. "Airline ground operations: Optimal schedule recovery with uncertain arrival times," Journal of Air Transport Management, Elsevier, vol. 92(C).
    16. Uğur Arıkan & Sinan Gürel & M. Selim Aktürk, 2017. "Flight Network-Based Approach for Integrated Airline Recovery with Cruise Speed Control," Transportation Science, INFORMS, vol. 51(4), pages 1259-1287, November.
    17. Kenan, Nabil & Jebali, Aida & Diabat, Ali, 2018. "The integrated aircraft routing problem with optional flights and delay considerations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 118(C), pages 355-375.
    18. Derui Wang & Yanfeng Wu & Jian-Qiang Hu & Miaomiao Liu & Peiwen Yu & Cheng Zhang & Yan Wu, 2019. "Flight Schedule Recovery: A Simulation-Based Approach," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 36(06), pages 1-19, December.
    19. Jon D. Petersen & Gustaf Sölveling & John-Paul Clarke & Ellis L. Johnson & Sergey Shebalov, 2012. "An Optimization Approach to Airline Integrated Recovery," Transportation Science, INFORMS, vol. 46(4), pages 482-500, November.
    20. Brouer, Berit D. & Dirksen, Jakob & Pisinger, David & Plum, Christian E.M. & Vaaben, Bo, 2013. "The Vessel Schedule Recovery Problem (VSRP) – A MIP model for handling disruptions in liner shipping," European Journal of Operational Research, Elsevier, vol. 224(2), pages 362-374.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:179:y:2023:i:c:s1366554523002831. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.