IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v162y2014i1d10.1007_s10957-012-0262-8.html
   My bibliography  Save this article

Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon

Author

Listed:
  • Hugo Cruz-Suárez

    (Benemérita Universidad Autónoma de Puebla)

  • Rocio Ilhuicatzi-Roldán

    (Benemérita Universidad Autónoma de Puebla)

  • Raúl Montes-de-Oca

    (Universidad Autónoma Metropolitana-Iztapalapa)

Abstract

This paper deals with Markov Decision Processes (MDPs) on Borel spaces with possibly unbounded costs. The criterion to be optimized is the expected total cost with a random horizon of infinite support. In this paper, it is observed that this performance criterion is equivalent to the expected total discounted cost with an infinite horizon and a varying-time discount factor. Then, the optimal value function and the optimal policy are characterized through some suitable versions of the Dynamic Programming Equation. Moreover, it is proved that the optimal value function of the optimal control problem with a random horizon can be bounded from above by the optimal value function of a discounted optimal control problem with a fixed discount factor. In this case, the discount factor is defined in an adequate way by the parameters introduced for the study of the optimal control problem with a random horizon. To illustrate the theory developed, a version of the Linear-Quadratic model with a random horizon and a Logarithm Consumption-Investment model are presented.

Suggested Citation

  • Hugo Cruz-Suárez & Rocio Ilhuicatzi-Roldán & Raúl Montes-de-Oca, 2014. "Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon," Journal of Optimization Theory and Applications, Springer, vol. 162(1), pages 329-346, July.
    • Handle: RePEc:spr:joptap:v:162:y:2014:i:1:d:10.1007_s10957-012-0262-8
    DOI: 10.1007/s10957-012-0262-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-012-0262-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-012-0262-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Levhari, David & Mirman, Leonard J, 1977. "Savings and Consumption with an Uncertain Horizon," Journal of Political Economy, University of Chicago Press, vol. 85(2), pages 265-281, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David González-Sánchez & Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa, 2019. "Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies," Dynamic Games and Applications, Springer, vol. 9(1), pages 103-121, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul, Satya & Dasgupta, Ajit K., 1989. "Inheritance and wealth inequality : The case of the Punjab," Journal of Development Economics, Elsevier, vol. 30(2), pages 301-324, April.
    2. Joan Costa-Font & Cristina Vilaplana-Prieto, 2022. "Biased survival expectations and behaviours: Does domain specific information matter?," Journal of Risk and Uncertainty, Springer, vol. 65(3), pages 285-317, December.
    3. Ramser, Hans Jürgen, 1977. "Lebenszyklustheorie des Sparens: Zum Stand der Theorie," Discussion Papers, Series I 101, University of Konstanz, Department of Economics.
    4. HILDEBRAND Vincent, 2001. "Wealth Accumulation of US Households: What do we learn from the SIPP data?," IRISS Working Paper Series 2001-01, IRISS at CEPS/INSTEAD.
    5. Gary Hansen & Selahattin Imrohoroglu, 2008. "Consumption over the Life Cycle: The Role of Annuities," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 11(3), pages 566-583, July.
    6. Richard M. Peck & Padmanabhan Srinagesh, 1986. "Time Preference and Uncertain Lifetime1," The American Economist, Sage Publications, vol. 30(2), pages 41-45, October.
    7. Bommier, Antoine & Leroux, Marie-Louise & Lozachmeur, Jean-Marie, 2011. "On the public economics of annuities with differential mortality," Journal of Public Economics, Elsevier, vol. 95(7), pages 612-623.
    8. R. Glenn Hubbard, 1987. "Uncertain Lifetimes, Pensions, and Individual Saving," NBER Chapters, in: Issues in Pension Economics, pages 175-210, National Bureau of Economic Research, Inc.
    9. Milevsky, Moshe A., 2020. "Swimming with wealthy sharks: longevity, volatility and the value of risk pooling," Journal of Pension Economics and Finance, Cambridge University Press, vol. 19(2), pages 217-246, April.
    10. Traian A Pirvu & Ulrich G Haussmann, 2007. "On Robust Utility Maximization," Papers math/0702727, arXiv.org.
    11. Paul Menchik, 1980. "The Importance of Material Inheritance: The Financial Link between Generations," NBER Chapters, in: Modeling the Distribution and Intergenerational Transmission of Wealth, pages 159-186, National Bureau of Economic Research, Inc.
    12. John P. Rust, 1989. "A Dynamic Programming Model of Retirement Behavior," NBER Chapters, in: The Economics of Aging, pages 359-404, National Bureau of Economic Research, Inc.
    13. Post, Thomas & Hanewald, Katja, 2013. "Longevity risk, subjective survival expectations, and individual saving behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 200-220.
    14. Boyer, M. Martin & Stentoft, Lars, 2013. "If we can simulate it, we can insure it: An application to longevity risk management," Insurance: Mathematics and Economics, Elsevier, vol. 52(1), pages 35-45.
    15. repec:hum:wpaper:sfb649dp2010-040 is not listed on IDEAS
    16. Siu Fai Leung, 1995. "A distinction between continuous-time and discrete-time models of uncertain lifetime," Economics Letters, Elsevier, vol. 47(3-4), pages 291-296, March.
    17. Sandra Freire, 2004. "Funeral Costs, Saving Behaviour and HIV/AIDS," Cahiers de la Maison des Sciences Economiques bla04092, Université Panthéon-Sorbonne (Paris 1).
    18. Daniel S. Hamermesh, 1985. "Expectations, Life Expectancy, and Economic Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 100(2), pages 389-408.
    19. Charles Yuji Horioka, 2002. "Are the Japanese Selfish, Altruistic or Dynastic?," The Japanese Economic Review, Japanese Economic Association, vol. 53(1), pages 26-54, March.
    20. Huang, H. & Milevsky, M.A. & Salisbury, T.S., 2017. "Retirement spending and biological age," Journal of Economic Dynamics and Control, Elsevier, vol. 84(C), pages 58-76.
    21. repec:dau:papers:123456789/5369 is not listed on IDEAS
    22. Gary D. Hansen, "undated". "Consumption over the Life Cycle: The Role of Annuities (with Selo Imrohoroglu)," UCLA Economics Online Papers 417, UCLA Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:162:y:2014:i:1:d:10.1007_s10957-012-0262-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.