IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v217y2012i2p342-350.html
   My bibliography  Save this article

Monotone optimal control for a class of Markov decision processes

Author

Listed:
  • Zhuang, Weifen
  • Li, Michael Z.F.

Abstract

This paper provides a unified framework to study monotone optimal control for a class of Markov decision processes through D-multimodularity. We demonstrate that each system in this class can be classified as either a substitution-type or a complement-type system according to the possible transition set, which can be used as a classification mechanism that integrates a variety of models in the literature. We develop a generic proof of the structural properties of both types of system. In particular, we show that D-multimodularity is a generally sufficient condition for monotone optimal control of different types of system in this class. With this unified theory, there is no need to pursue each problem ad hoc and the structural properties of this class of MDPs follow with ease.

Suggested Citation

  • Zhuang, Weifen & Li, Michael Z.F., 2012. "Monotone optimal control for a class of Markov decision processes," European Journal of Operational Research, Elsevier, vol. 217(2), pages 342-350.
    • Handle: RePEc:eee:ejores:v:217:y:2012:i:2:p:342-350
    DOI: 10.1016/j.ejor.2011.09.021
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221711008496
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2011.09.021?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Janakiram Subramanian & Shaler Stidham & Conrad J. Lautenbacher, 1999. "Airline Yield Management with Overbooking, Cancellations, and No-Shows," Transportation Science, INFORMS, vol. 33(2), pages 147-167, May.
    2. Guillermo Gallego & Garrett van Ryzin, 1994. "Optimal Dynamic Pricing of Inventories with Stochastic Demand over Finite Horizons," Management Science, INFORMS, vol. 40(8), pages 999-1020, August.
    3. Cheng-Shang Chang & David D. Yao, 1993. "Rearrangement, Majorization and Stochastic Scheduling," Mathematics of Operations Research, INFORMS, vol. 18(3), pages 658-684, August.
    4. Albert Y. Ha, 1997. "Inventory Rationing in a Make-to-Stock Production System with Several Demand Classes and Lost Sales," Management Science, INFORMS, vol. 43(8), pages 1093-1103, August.
    5. Tak C. Lee & Marvin Hersh, 1993. "A Model for Dynamic Airline Seat Inventory Control with Multiple Seat Bookings," Transportation Science, INFORMS, vol. 27(3), pages 252-265, August.
    6. Wen Zhao & Yu-Sheng Zheng, 2000. "Optimal Dynamic Pricing for Perishable Assets with Nonhomogeneous Demand," Management Science, INFORMS, vol. 46(3), pages 375-388, March.
    7. James E. Smith & Kevin F. McCardle, 2002. "Structural Properties of Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 50(5), pages 796-809, October.
    8. Ghoneim, Hussein A. & Stidham, Shaler, 1985. "Control of arrivals to two queues in series," European Journal of Operational Research, Elsevier, vol. 21(3), pages 399-409, September.
    9. Sergei V. Savin & Morris A. Cohen & Noah Gans & Ziv Katalan, 2005. "Capacity Management in Rental Businesses with Two Customer Bases," Operations Research, INFORMS, vol. 53(4), pages 617-631, August.
    10. Paul Zipkin, 2008. "On the Structure of Lost-Sales Inventory Models," Operations Research, INFORMS, vol. 56(4), pages 937-944, August.
    11. Gabriel R. Bitran & Susana V. Mondschein, 1997. "Periodic Pricing of Seasonal Products in Retailing," Management Science, INFORMS, vol. 43(1), pages 64-79, January.
    12. Kazuo Murota, 2005. "Note on Multimodularity and L-Convexity," Mathematics of Operations Research, INFORMS, vol. 30(3), pages 658-661, August.
    13. Morton, Alec, 2006. "Structural properties of network revenue management models: an economic perspective," LSE Research Online Documents on Economics 2563, London School of Economics and Political Science, LSE Library.
    14. Linda V. Green & Sergei Savin & Ben Wang, 2006. "Managing Patient Service in a Diagnostic Medical Facility," Operations Research, INFORMS, vol. 54(1), pages 11-25, February.
    15. Albert Y. Ha, 1997. "Optimal Dynamic Scheduling Policy for a Make-To-Stock Production System," Operations Research, INFORMS, vol. 45(1), pages 42-53, February.
    16. Diwakar Gupta & Lei Wang, 2008. "Revenue Management for a Primary-Care Clinic in the Presence of Patient Choice," Operations Research, INFORMS, vol. 56(3), pages 576-592, June.
    17. Hui Zhao & Jennifer K. Ryan & Vinayak Deshpande, 2008. "Optimal Dynamic Production and Inventory Transshipment Policies for a Two-Location Make-to-Stock System," Operations Research, INFORMS, vol. 56(2), pages 400-410, April.
    18. Bruce Hajek, 1985. "Extremal Splittings of Point Processes," Mathematics of Operations Research, INFORMS, vol. 10(4), pages 543-556, November.
    19. Woonghee Tim Huh & Ganesh Janakiraman, 2010. "On the Optimal Policy Structure in Serial Inventory Systems with Lost Sales," Operations Research, INFORMS, vol. 58(2), pages 486-491, April.
    20. Shaler Stidham, 2002. "Analysis, Design, and Control of Queueing Systems," Operations Research, INFORMS, vol. 50(1), pages 197-216, February.
    21. Donald M. Topkis, 1978. "Minimizing a Submodular Function on a Lattice," Operations Research, INFORMS, vol. 26(2), pages 305-321, April.
    22. Peng-Sheng You, 1999. "Dynamic Pricing in Airline Seat Management for Flights with Multiple Flight Legs," Transportation Science, INFORMS, vol. 33(2), pages 192-206, May.
    23. Paul Glasserman & David D. Yao, 1994. "Monotone Optimal Control of Permutable GSMPs," Mathematics of Operations Research, INFORMS, vol. 19(2), pages 449-476, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David Sayah & Stefan Irnich, 2019. "Optimal booking control in revenue management with two substitutable resources," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 89(2), pages 189-222, April.
    2. Qing Li & Peiwen Yu, 2014. "Multimodularity and Its Applications in Three Stochastic Dynamic Inventory Problems," Manufacturing & Service Operations Management, INFORMS, vol. 16(3), pages 455-463, July.
    3. Legros, Benjamin & Jouini, Oualid, 2019. "On the scheduling of operations in a chat contact center," European Journal of Operational Research, Elsevier, vol. 274(1), pages 303-316.
    4. Legros, Benjamin, 2019. "Dynamic repositioning strategy in a bike-sharing system; how to prioritize and how to rebalance a bike station," European Journal of Operational Research, Elsevier, vol. 272(2), pages 740-753.
    5. David Sayah & Stefan Irnich, 2016. "Optimal Booking Control in Airline Revenue Management with Two Flights and Flexible Products," Working Papers 1604, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    6. Samuel Vercraene & Jean-Philippe Gayon & Fikri Karaesmen, 2018. "Effects of System Parameters on the Optimal Cost and Policy in a Class of Multidimensional Queueing Control Problems," Operations Research, INFORMS, vol. 66(1), pages 150-162, January.
    7. Miehling, Erik & Teneketzis, Demosthenis, 2020. "Monotonicity properties for two-action partially observable Markov decision processes on partially ordered spaces," European Journal of Operational Research, Elsevier, vol. 282(3), pages 936-944.
    8. Keumseok Kang & J. George Shanthikumar & Kemal Altinkemer, 2016. "Postponable Acceptance and Assignment: A Stochastic Dynamic Programming Approach," Manufacturing & Service Operations Management, INFORMS, vol. 18(4), pages 493-508, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriel Bitran & René Caldentey, 2003. "An Overview of Pricing Models for Revenue Management," Manufacturing & Service Operations Management, INFORMS, vol. 5(3), pages 203-229, August.
    2. Alec Morton, 2006. "Structural properties of network revenue management models: An economic perspective," Naval Research Logistics (NRL), John Wiley & Sons, vol. 53(8), pages 748-760, December.
    3. Kyle Y. Lin, 2004. "A sequential dynamic pricing model and its applications," Naval Research Logistics (NRL), John Wiley & Sons, vol. 51(4), pages 501-521, June.
    4. Hans-Jörg Schütz & Rainer Kolisch, 2013. "Capacity allocation for demand of different customer-product-combinations with cancellations, no-shows, and overbooking when there is a sequential delivery of service," Annals of Operations Research, Springer, vol. 206(1), pages 401-423, July.
    5. Qing Li & Peiwen Yu, 2014. "Multimodularity and Its Applications in Three Stochastic Dynamic Inventory Problems," Manufacturing & Service Operations Management, INFORMS, vol. 16(3), pages 455-463, July.
    6. Pak, K. & Piersma, N., 2002. "airline revenue management," ERIM Report Series Research in Management ERS-2002-12-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    7. Pak, K. & Piersma, N., 2002. "Airline revenue management: an overview of OR techniques 1982-2001," Econometric Institute Research Papers EI 2002-03, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    8. Lin, Kyle Y. & Sibdari, Soheil Y., 2009. "Dynamic price competition with discrete customer choices," European Journal of Operational Research, Elsevier, vol. 197(3), pages 969-980, September.
    9. Qing Ding & Panos Kouvelis & Joseph M. Milner, 2006. "Dynamic Pricing Through Discounts for Optimizing Multiple-Class Demand Fulfillment," Operations Research, INFORMS, vol. 54(1), pages 169-183, February.
    10. Syed Asif Raza & Rafi Ashrafi & Ali Akgunduz, 2020. "A bibliometric analysis of revenue management in airline industry," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 19(6), pages 436-465, December.
    11. William L. Cooper, 2002. "Asymptotic Behavior of an Allocation Policy for Revenue Management," Operations Research, INFORMS, vol. 50(4), pages 720-727, August.
    12. Van-Anh Truong, 2015. "Optimal Advance Scheduling," Management Science, INFORMS, vol. 61(7), pages 1584-1597, July.
    13. Tong Wang & Xiting Gong & Sean X. Zhou, 2017. "Dynamic Inventory Management with Total Minimum Order Commitments and Two Supply Options," Operations Research, INFORMS, vol. 65(5), pages 1285-1302, October.
    14. Liping Zhou & Na Geng & Zhibin Jiang & Shan Jiang, 2022. "Integrated Multiresource Capacity Planning and Multitype Patient Scheduling," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 129-149, January.
    15. Jeffrey I. McGill & Garrett J. van Ryzin, 1999. "Revenue Management: Research Overview and Prospects," Transportation Science, INFORMS, vol. 33(2), pages 233-256, May.
    16. Wen Zhao & Yu-Sheng Zheng, 2000. "Optimal Dynamic Pricing for Perishable Assets with Nonhomogeneous Demand," Management Science, INFORMS, vol. 46(3), pages 375-388, March.
    17. Guillermo Gallego & Michael Z. F. Li & Yan Liu, 2020. "Dynamic Nonlinear Pricing of Inventories over Finite Sales Horizons," Operations Research, INFORMS, vol. 68(3), pages 655-670, May.
    18. J. Neil Bearden & Ryan O. Murphy & Amnon Rapoport, 2008. "Decision Biases in Revenue Management: Some Behavioral Evidence," Manufacturing & Service Operations Management, INFORMS, vol. 10(4), pages 625-636, June.
    19. Schütz, Hans-Jörg & Kolisch, Rainer, 2012. "Approximate dynamic programming for capacity allocation in the service industry," European Journal of Operational Research, Elsevier, vol. 218(1), pages 239-250.
    20. Yalç{i}n Akçay & Harihara Prasad Natarajan & Susan H. Xu, 2010. "Joint Dynamic Pricing of Multiple Perishable Products Under Consumer Choice," Management Science, INFORMS, vol. 56(8), pages 1345-1361, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:217:y:2012:i:2:p:342-350. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.