IDEAS home Printed from https://ideas.repec.org/a/pal/jorapm/v20y2021i5d10.1057_s41272-020-00275-x.html
   My bibliography  Save this article

Deep reinforcement learning in seat inventory control problem: an action generation approach

Author

Listed:
  • Neda Etebari Alamdari

    (MAGI, Polytechnique Montreal)

  • Gilles Savard

    (MAGI, Polytechnique Montreal)

Abstract

Nowadays, firms intend to use customer choice-based models instead of an independent demand paradigm to generate more revenue. In this paper, we address choice-based seat inventory control problem with stochastic demand using a deep reinforcement learning technique named Deep Q-Network (DQN). DQN can naturally address large state space problems with its integrated function approximation. However, it becomes intractable in the case of large discrete action space. To address this issue, we propose an Action Generation (AGen) algorithm. AGen is a greedy heuristic algorithm designed to be integrated into DQN to overcome the complexity of the original problem. It aims to greedily generate a set of “effective” actions to replace the original action space. This leads to the main achievement of this study which is to dramatically decrease the complexity of the solution method without negatively affecting its performance in a large-scale choice-based seat inventory allocation problem.

Suggested Citation

  • Neda Etebari Alamdari & Gilles Savard, 2021. "Deep reinforcement learning in seat inventory control problem: an action generation approach," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 20(5), pages 566-579, October.
  • Handle: RePEc:pal:jorapm:v:20:y:2021:i:5:d:10.1057_s41272-020-00275-x
    DOI: 10.1057/s41272-020-00275-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41272-020-00275-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1057/s41272-020-00275-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Meissner, Joern & Strauss, Arne, 2012. "Network revenue management with inventory-sensitive bid prices and customer choice," European Journal of Operational Research, Elsevier, vol. 216(2), pages 459-468.
    2. Hosseinalifam, M. & Marcotte, P. & Savard, G., 2016. "A new bid price approach to dynamic resource allocation in network revenue management," European Journal of Operational Research, Elsevier, vol. 255(1), pages 142-150.
    3. Strauss, Arne K. & Klein, Robert & Steinhardt, Claudius, 2018. "A review of choice-based revenue management: Theory and methods," European Journal of Operational Research, Elsevier, vol. 271(2), pages 375-387.
    4. Juan M. Chaneton & Gustavo Vulcano, 2011. "Computing Bid Prices for Revenue Management Under Customer Choice Behavior," Manufacturing & Service Operations Management, INFORMS, vol. 13(4), pages 452-470, October.
    5. Nicolas Bondoux & Anh Quan Nguyen & Thomas Fiig & Rodrigo Acuna-Agost, 2020. "Reinforcement learning applied to airline revenue management," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 19(5), pages 332-348, October.
    6. Juan José Miranda Bront & Isabel Méndez-Díaz & Gustavo Vulcano, 2009. "A Column Generation Algorithm for Choice-Based Network Revenue Management," Operations Research, INFORMS, vol. 57(3), pages 769-784, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sumit Kunnumkal & Kalyan Talluri, 2019. "Choice Network Revenue Management Based on New Tractable Approximations," Transportation Science, INFORMS, vol. 53(6), pages 1591-1608, November.
    2. Barbier, Thibault & Anjos, Miguel F. & Cirinei, Fabien & Savard, Gilles, 2020. "Product-closing approximation for ranking-based choice network revenue management," European Journal of Operational Research, Elsevier, vol. 286(3), pages 1002-1017.
    3. Paat Rusmevichientong & Huseyin Topaloglu, 2012. "Robust Assortment Optimization in Revenue Management Under the Multinomial Logit Choice Model," Operations Research, INFORMS, vol. 60(4), pages 865-882, August.
    4. Sebastian Koch & Jochen Gönsch & Michael Hassler & Robert Klein, 2016. "Practical decision rules for risk-averse revenue management using simulation-based optimization," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 15(6), pages 468-487, December.
    5. Koch, Sebastian & Klein, Robert, 2020. "Route-based approximate dynamic programming for dynamic pricing in attended home delivery," European Journal of Operational Research, Elsevier, vol. 287(2), pages 633-652.
    6. Nicolas Houy & François Le Grand, 2015. "The Monte Carlo first-come-first-served heuristic for network revenue management," Working Papers halshs-01155698, HAL.
    7. Fleckenstein, David & Klein, Robert & Steinhardt, Claudius, 2023. "Recent advances in integrating demand management and vehicle routing: A methodological review," European Journal of Operational Research, Elsevier, vol. 306(2), pages 499-518.
    8. Dan Zhang, 2011. "An Improved Dynamic Programming Decomposition Approach for Network Revenue Management," Manufacturing & Service Operations Management, INFORMS, vol. 13(1), pages 35-52, April.
    9. Anton J. Kleywegt & Hongzhang Shao, 2022. "Revenue Management Under the Markov Chain Choice Model with Joint Price and Assortment Decisions," Papers 2204.04774, arXiv.org.
    10. Alexander Erdelyi & Huseyin Topaloglu, 2010. "A Dynamic Programming Decomposition Method for Making Overbooking Decisions Over an Airline Network," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 443-456, August.
    11. Laumer, Simon & Barz, Christiane, 2023. "Reductions of non-separable approximate linear programs for network revenue management," European Journal of Operational Research, Elsevier, vol. 309(1), pages 252-270.
    12. Sumit Kunnumkal & Kalyan Talluri, 2016. "On a Piecewise-Linear Approximation for Network Revenue Management," Mathematics of Operations Research, INFORMS, vol. 41(1), pages 72-91, February.
    13. Stefanus Jasin & Sunil Kumar, 2012. "A Re-Solving Heuristic with Bounded Revenue Loss for Network Revenue Management with Customer Choice," Mathematics of Operations Research, INFORMS, vol. 37(2), pages 313-345, May.
    14. Dan Zhang & Larry Weatherford, 2017. "Dynamic Pricing for Network Revenue Management: A New Approach and Application in the Hotel Industry," INFORMS Journal on Computing, INFORMS, vol. 29(1), pages 18-35, February.
    15. Sumit Kunnumkal & Kalyan Talluri, 2016. "Technical Note—A Note on Relaxations of the Choice Network Revenue Management Dynamic Program," Operations Research, INFORMS, vol. 64(1), pages 158-166, February.
    16. Rodrigo Acuna-Agost & Eoin Thomas & Alix Lhéritier, 2021. "Price elasticity estimation for deep learning-based choice models: an application to air itinerary choices," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 20(3), pages 213-226, June.
    17. Wuyang Yuan & Lei Nie & Xin Wu & Huiling Fu, 2018. "A dynamic bid price approach for the seat inventory control problem in railway networks with consideration of passenger transfer," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-23, August.
    18. Paat Rusmevichientong & Zuo-Jun Max Shen & David B. Shmoys, 2010. "Dynamic Assortment Optimization with a Multinomial Logit Choice Model and Capacity Constraint," Operations Research, INFORMS, vol. 58(6), pages 1666-1680, December.
    19. Kalyan Talluri, 2014. "New Formulations for Choice Network Revenue Management," INFORMS Journal on Computing, INFORMS, vol. 26(2), pages 401-413, May.
    20. Sumit Kunnumkal & Kalyan Talluri, 2019. "A strong Lagrangian relaxation for general discrete-choice network revenue management," Computational Optimization and Applications, Springer, vol. 73(1), pages 275-310, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:jorapm:v:20:y:2021:i:5:d:10.1057_s41272-020-00275-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.palgrave.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.