IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v37y1989i3p474-477.html
   My bibliography  Save this article

Randomized and Past-Dependent Policies for Markov Decision Processes with Multiple Constraints

Author

Listed:
  • Keith W. Ross

    (University of Pennsylvania, Philadelphia, Pennsylvania)

Abstract

The Markov decision problem of locating a policy to maximize the long-run average reward subject to K long-run average cost constraints is considered. It is assumed that the state and action spaces are finite and the law of motion is unichain, that is, every pure policy gives rise to a Markov chain with one recurrent class. It is first proved that there exists an optimal stationary policy with a degree of randomization no greater than K ; consequently, it is never necessary to randomize in more than K states. A linear program produces the optimal policy with limited randomization. For the special case of a single constraint, we also address the problem of finding optimal nonrandomized, but nonstationary, policies. We show that a round-robin type policy is optimal, and conjecture the same for a steering policy that depends on the entire past history of the process, but whose implementation requires essentially no more storage than that of a pure policy.

Suggested Citation

  • Keith W. Ross, 1989. "Randomized and Past-Dependent Policies for Markov Decision Processes with Multiple Constraints," Operations Research, INFORMS, vol. 37(3), pages 474-477, June.
  • Handle: RePEc:inm:oropre:v:37:y:1989:i:3:p:474-477
    DOI: 10.1287/opre.37.3.474
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.37.3.474
    Download Restriction: no

    File URL: https://libkey.io/10.1287/opre.37.3.474?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vladimir Ejov & Jerzy A. Filar & Minh-Tuan Nguyen, 2004. "Hamiltonian Cycles and Singularly Perturbed Markov Chains," Mathematics of Operations Research, INFORMS, vol. 29(1), pages 114-131, February.
    2. Daniel F. Silva & Bo Zhang & Hayriye Ayhan, 2020. "Optimal strategies for managing complex authentication systems," Annals of Operations Research, Springer, vol. 293(1), pages 317-342, October.
    3. Ohlmann, Jeffrey W. & Bean, James C., 2009. "Resource-constrained management of heterogeneous assets with stochastic deterioration," European Journal of Operational Research, Elsevier, vol. 199(1), pages 198-208, November.
    4. Dmitry Krass & O. J. Vrieze, 2002. "Achieving Target State-Action Frequencies in Multichain Average-Reward Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 27(3), pages 545-566, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:37:y:1989:i:3:p:474-477. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.