IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v35y1987i6p875-883.html
   My bibliography  Save this article

A New Optimality Criterion for Nonhomogeneous Markov Decision Processes

Author

Listed:
  • Wallace J. Hopp

    (Northwestern University, Evanston, Illinois)

  • James C. Bean

    (The University of Michigan, Ann Arbor, Michigan)

  • Robert L. Smith

    (The University of Michigan, Ann Arbor, Michigan)

Abstract

We propose a new definition of optimality for nonhomogeneous Markov decision processes called periodic forecast horizon (PFH) optimality. Using measures of discounting and ergodicity, we establish conditions under which PFH-optimal strategies exist and PFH optimality implies the more conventional notions of α-optimality and average optimality. Finally, we use this definition of optimality as the basis for a direct development of forecast horizon results for discounted and undiscounted nonhomogeneous Markov decision processes.

Suggested Citation

  • Wallace J. Hopp & James C. Bean & Robert L. Smith, 1987. "A New Optimality Criterion for Nonhomogeneous Markov Decision Processes," Operations Research, INFORMS, vol. 35(6), pages 875-883, December.
    • Handle: RePEc:inm:oropre:v:35:y:1987:i:6:p:875-883
    DOI: 10.1287/opre.35.6.875
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.35.6.875
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.35.6.875?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sampath Rajagopalan & Medini R. Singh & Thomas E. Morton, 1998. "Capacity Expansion and Replacement in Growing Markets with Uncertain Technological Breakthroughs," Management Science, INFORMS, vol. 44(1), pages 12-30, January.
    2. Iida, Tetsuo, 2001. "The infinite horizon non-stationary stochastic multi-echelon inventory problem and near-myopic policies," European Journal of Operational Research, Elsevier, vol. 134(3), pages 525-539, November.
    3. Torpong Cheevaprawatdomrong & Irwin E. Schochetman & Robert L. Smith & Alfredo Garcia, 2007. "Solution and Forecast Horizons for Infinite-Horizon Nonhomogeneous Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 32(1), pages 51-72, February.
    4. Archis Ghate & Dushyant Sharma & Robert L. Smith, 2010. "A Shadow Simplex Method for Infinite Linear Programs," Operations Research, INFORMS, vol. 58(4-part-1), pages 865-877, August.
    5. William P. Cross & H. Edwin Romeijn & Robert L. Smith, 1998. "Approximating Extreme Points of Infinite Dimensional Convex Sets," Mathematics of Operations Research, INFORMS, vol. 23(2), pages 433-442, May.
    6. Iida, Tetsuo, 1999. "The infinite horizon non-stationary stochastic inventory problem: Near myopic policies and weak ergodicity," European Journal of Operational Research, Elsevier, vol. 116(2), pages 405-422, July.
    7. Kang, Yuncheol & Sawyer, Amy M. & Griffin, Paul M. & Prabhu, Vittaldas V., 2016. "Modelling adherence behaviour for the treatment of obstructive sleep apnoea," European Journal of Operational Research, Elsevier, vol. 249(3), pages 1005-1013.
    8. Khanh T.P. Nguyen & Thomas Yeung & Bruno Castanier, 2017. "Acquisition of new technology information for maintenance and replacement policies," International Journal of Production Research, Taylor & Francis Journals, vol. 55(8), pages 2212-2231, April.
    9. Archis Ghate & Robert L. Smith, 2013. "A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes," Operations Research, INFORMS, vol. 61(2), pages 413-425, April.
    10. Allise O. Wachs & Irwin E. Schochetman & Robert L. Smith, 2011. "Average Optimality in Nonhomogeneous Infinite Horizon Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 36(1), pages 147-164, February.
    11. Xianping Guo & Jianyong Liu & Ke Liu, 2000. "Nonhomogeneous Markov Decision Processes with Borel State Space---The Average Criterion with Nonuniformly Bounded Rewards," Mathematics of Operations Research, INFORMS, vol. 25(4), pages 667-678, November.
    12. Suresh Chand & Vernon Ning Hsu & Suresh Sethi, 2002. "Forecast, Solution, and Rolling Horizons in Operations Management Problems: A Classified Bibliography," Manufacturing & Service Operations Management, INFORMS, vol. 4(1), pages 25-43, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:35:y:1987:i:6:p:875-883. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.