IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v162y2014i1d10.1007_s10957-012-0262-8.html
   My bibliography  Save this article

Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon

Author

Listed:
  • Hugo Cruz-Suárez

    (Benemérita Universidad Autónoma de Puebla)

  • Rocio Ilhuicatzi-Roldán

    (Benemérita Universidad Autónoma de Puebla)

  • Raúl Montes-de-Oca

    (Universidad Autónoma Metropolitana-Iztapalapa)

Abstract

This paper deals with Markov Decision Processes (MDPs) on Borel spaces with possibly unbounded costs. The criterion to be optimized is the expected total cost with a random horizon of infinite support. In this paper, it is observed that this performance criterion is equivalent to the expected total discounted cost with an infinite horizon and a varying-time discount factor. Then, the optimal value function and the optimal policy are characterized through some suitable versions of the Dynamic Programming Equation. Moreover, it is proved that the optimal value function of the optimal control problem with a random horizon can be bounded from above by the optimal value function of a discounted optimal control problem with a fixed discount factor. In this case, the discount factor is defined in an adequate way by the parameters introduced for the study of the optimal control problem with a random horizon. To illustrate the theory developed, a version of the Linear-Quadratic model with a random horizon and a Logarithm Consumption-Investment model are presented.

Suggested Citation

  • Hugo Cruz-Suárez & Rocio Ilhuicatzi-Roldán & Raúl Montes-de-Oca, 2014. "Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon," Journal of Optimization Theory and Applications, Springer, vol. 162(1), pages 329-346, July.
    • Handle: RePEc:spr:joptap:v:162:y:2014:i:1:d:10.1007_s10957-012-0262-8
    DOI: 10.1007/s10957-012-0262-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-012-0262-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-012-0262-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Levhari, David & Mirman, Leonard J, 1977. "Savings and Consumption with an Uncertain Horizon," Journal of Political Economy, University of Chicago Press, vol. 85(2), pages 265-281, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David González-Sánchez & Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa, 2019. "Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies," Dynamic Games and Applications, Springer, vol. 9(1), pages 103-121, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul, Satya & Dasgupta, Ajit K., 1989. "Inheritance and wealth inequality : The case of the Punjab," Journal of Development Economics, Elsevier, vol. 30(2), pages 301-324, April.
    2. Joan Costa-Font & Cristina Vilaplana-Prieto, 2022. "Biased survival expectations and behaviours: Does domain specific information matter?," Journal of Risk and Uncertainty, Springer, vol. 65(3), pages 285-317, December.
    3. Huang, H. & Milevsky, M.A. & Salisbury, T.S., 2017. "Retirement spending and biological age," Journal of Economic Dynamics and Control, Elsevier, vol. 84(C), pages 58-76.
    4. Gahramanov, Emin, 2013. "Survival misperception, time inconsistency, and implications for life-cycle saving and welfare," Economic Modelling, Elsevier, vol. 32(C), pages 539-550.
    5. Huang, Huaxiong & Milevsky, Moshe A. & Salisbury, Thomas S., 2012. "Optimal retirement consumption with a stochastic force of mortality," Insurance: Mathematics and Economics, Elsevier, vol. 51(2), pages 282-291.
    6. Ramser, Hans Jürgen, 1977. "Lebenszyklustheorie des Sparens: Zum Stand der Theorie," Discussion Papers, Series I 101, University of Konstanz, Department of Economics.
    7. HILDEBRAND Vincent, 2001. "Wealth Accumulation of US Households: What do we learn from the SIPP data?," IRISS Working Paper Series 2001-01, IRISS at CEPS/INSTEAD.
    8. Gary Hansen & Selahattin Imrohoroglu, 2008. "Consumption over the Life Cycle: The Role of Annuities," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 11(3), pages 566-583, July.
    9. Richard M. Peck & Padmanabhan Srinagesh, 1986. "Time Preference and Uncertain Lifetime1," The American Economist, Sage Publications, vol. 30(2), pages 41-45, October.
    10. Mervyn A. King, 1983. "The Economics of Saving," NBER Working Papers 1247, National Bureau of Economic Research, Inc.
    11. Bommier, Antoine & Leroux, Marie-Louise & Lozachmeur, Jean-Marie, 2011. "On the public economics of annuities with differential mortality," Journal of Public Economics, Elsevier, vol. 95(7), pages 612-623.
    12. R. Glenn Hubbard, 1987. "Uncertain Lifetimes, Pensions, and Individual Saving," NBER Chapters, in: Issues in Pension Economics, pages 175-210, National Bureau of Economic Research, Inc.
    13. Milevsky, Moshe A., 2020. "Swimming with wealthy sharks: longevity, volatility and the value of risk pooling," Journal of Pension Economics and Finance, Cambridge University Press, vol. 19(2), pages 217-246, April.
    14. Sanna Nivakoski, 2020. "Wealth and the effect of subjective survival probability," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(2), pages 633-670, April.
    15. Hamermesh, Daniel S, 1984. "Consumption during Retirement: The Missing Link in the Life Cycle," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 1-7, February.
    16. Traian A Pirvu & Ulrich G Haussmann, 2007. "On Robust Utility Maximization," Papers math/0702727, arXiv.org.
    17. Charles Yuji Horioka, 2002. "Are the Japanese Selfish, Altruistic or Dynastic?," The Japanese Economic Review, Japanese Economic Association, vol. 53(1), pages 26-54, March.
    18. Mervyn A. King & Louis Dicks-Mireaux, 1981. "Asset Holdings and the Life Cycle," NBER Working Papers 0614, National Bureau of Economic Research, Inc.
    19. Daniel S. Hamermesh, 1985. "Expectations, Life Expectancy, and Economic Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 100(2), pages 389-408.
    20. Pablo Garcia-Sanchez & Olivier Pierrard, 2023. "Uncertain Lifetime, Health Investment And Welfare," LIDAM Discussion Papers IRES 2023020, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:162:y:2014:i:1:d:10.1007_s10957-012-0262-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.