IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v287y2020i2d10.1007_s10479-018-2778-2.html
   My bibliography  Save this article

Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting

Author

Listed:
  • Łukasz Balbus

    (University of Zielona Góra)

  • Anna Jaśkiewicz

    (Wrocław University of Science and Technology)

  • Andrzej S. Nowak

    (University of Zielona Góra)

Abstract

We study a discrete-time non-stationary decision model in which the preferences of the decision maker change over time and are described by quasi-hyperbolic discounting. A time-consistent optimal solution in this model corresponds with a Markov perfect equilibrium in a stochastic game with uncountable state space played by countably many short-lived players. We show that Markov perfect equilibria may be constructed using a generalized policy iteration algorithm. This method is in part inspired by the fundamental works of Mertens and Parthasarathy (in: Raghavan, Ferguson, Parthasarathy, Vrieze (eds) Stochastic games and related topics, Kluwer Academic Publishers, Dordrecht, 1991; in: Neyman, Sorin (eds) Stochastic games and applications, Academic Publishers, Dordrecht, 2003) devoted to subgame perfect equilibria in standard n-person discounted stochastic games. If the one-period utilities and transition probabilities are independent of time, we obtain on new existence results on stationary Markov perfect equilibria in the models with unbounded from above utilities.

Suggested Citation

  • Łukasz Balbus & Anna Jaśkiewicz & Andrzej S. Nowak, 2020. "Markov perfect equilibria in a dynamic decision model with quasi-hyperbolic discounting," Annals of Operations Research, Springer, vol. 287(2), pages 573-591, April.
    • Handle: RePEc:spr:annopr:v:287:y:2020:i:2:d:10.1007_s10479-018-2778-2
    DOI: 10.1007/s10479-018-2778-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-018-2778-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-018-2778-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Harris, Christopher & Laibson, David, 2001. "Dynamic Choices of Hyperbolic Consumers," Econometrica, Econometric Society, vol. 69(4), pages 935-957, July.
    2. Jaśkiewicz, Anna & Nowak, Andrzej S., 2014. "Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models," Journal of Economic Theory, Elsevier, vol. 151(C), pages 411-447.
    3. Josef Haunschmied & Vladimir M. Veliov & Stefan Wrzaczek (ed.), 2014. "Dynamic Games in Economics," Dynamic Modeling and Econometrics in Economics and Finance, Springer, edition 127, number 978-3-642-54248-0, May.
    4. E. S. Phelps & R. A. Pollak, 1968. "On Second-Best National Saving and Game-Equilibrium Growth," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 35(2), pages 185-199.
    5. José Luis Montiel Olea & Tomasz Strzalecki, 2014. "Axiomatization and Measurement of Quasi-Hyperbolic Discounting," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1449-1499.
    6. Anna Jaśkiewicz & Andrzej Nowak, 2011. "Stochastic Games with Unbounded Payoffs: Applications to Robust Control in Economics," Dynamic Games and Applications, Springer, vol. 1(2), pages 253-279, June.
    7. Maskin, Eric & Tirole, Jean, 2001. "Markov Perfect Equilibrium: I. Observable Actions," Journal of Economic Theory, Elsevier, vol. 100(2), pages 191-219, October.
    8. Karp, Larry, 2005. "Global warming and hyperbolic discounting," Journal of Public Economics, Elsevier, vol. 89(2-3), pages 261-282, February.
    9. Mertens, J.-F. & Parthasarathy, T., 1991. "Nonzero-sum stochastic games," LIDAM Reprints CORE 912, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    10. Łukasz Balbus & Anna Jaśkiewicz & Andrzej S. Nowak, 2015. "Existence of Stationary Markov Perfect Equilibria in Stochastic Altruistic Growth Economies," Journal of Optimization Theory and Applications, Springer, vol. 165(1), pages 295-315, April.
    11. Wolfgang Leininger, 1986. "The Existence of Perfect Equilibria in a Model of Growth with Altruism between Generations," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(3), pages 349-367.
    12. Bezalel Peleg & Menahem E. Yaari, 1973. "On the Existence of a Consistent Course of Action when Tastes are Changing," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 40(3), pages 391-401.
    13. Alain Haurie, 2005. "A Multigenerational Game Model to Analyze Sustainable Development," Annals of Operations Research, Springer, vol. 137(1), pages 369-386, July.
    14. Charalambos D. Aliprantis & Kim C. Border, 2006. "Infinite Dimensional Analysis," Springer Books, Springer, edition 0, number 978-3-540-29587-7, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Balbus, Łukasz & Reffett, Kevin & Woźny, Łukasz, 2022. "Time-consistent equilibria in dynamic models with recursive payoffs and behavioral discounting," Journal of Economic Theory, Elsevier, vol. 204(C).
    2. Nicole Bauerle & Anna Ja'skiewicz, 2015. "Stochastic Optimal Growth Model with Risk Sensitive Preferences," Papers 1509.05638, arXiv.org.
    3. Anna Jaśkiewicz & Andrzej S. Nowak, 2021. "Markov decision processes with quasi-hyperbolic discounting," Finance and Stochastics, Springer, vol. 25(2), pages 189-229, April.
    4. Abdellatif Semmouri & Mostafa Jourhmane & Zineb Belhallaj, 2020. "Discounted Markov decision processes with fuzzy costs," Annals of Operations Research, Springer, vol. 295(2), pages 769-786, December.
    5. Jaśkiewicz, Anna & Nowak, Andrzej S., 2022. "A note on topological aspects in dynamic games of resource extraction and economic growth theory," Games and Economic Behavior, Elsevier, vol. 131(C), pages 264-274.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anna Jaśkiewicz & Andrzej S. Nowak, 2021. "Markov decision processes with quasi-hyperbolic discounting," Finance and Stochastics, Springer, vol. 25(2), pages 189-229, April.
    2. Balbus, Łukasz & Jaśkiewicz, Anna & Nowak, Andrzej S., 2016. "Non-paternalistic intergenerational altruism revisited," Journal of Mathematical Economics, Elsevier, vol. 63(C), pages 27-33.
    3. Jaśkiewicz, Anna & Nowak, Andrzej S., 2014. "Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models," Journal of Economic Theory, Elsevier, vol. 151(C), pages 411-447.
    4. Łukasz Balbus & Anna Jaśkiewicz & Andrzej S. Nowak, 2015. "Existence of Stationary Markov Perfect Equilibria in Stochastic Altruistic Growth Economies," Journal of Optimization Theory and Applications, Springer, vol. 165(1), pages 295-315, April.
    5. Balbus, Łukasz & Reffett, Kevin & Woźny, Łukasz, 2018. "On uniqueness of time-consistent Markov policies for quasi-hyperbolic consumers under uncertainty," Journal of Economic Theory, Elsevier, vol. 176(C), pages 293-310.
    6. Lilia Maliar & Serguei Maliar, 2016. "Ruling Out Multiplicity of Smooth Equilibria in Dynamic Games: A Hyperbolic Discounting Example," Dynamic Games and Applications, Springer, vol. 6(2), pages 243-261, June.
    7. Balbus, Łukasz & Jaśkiewicz, Anna & Nowak, Andrzej S., 2015. "Stochastic bequest games," Games and Economic Behavior, Elsevier, vol. 90(C), pages 247-256.
    8. Łukasz Balbus & Anna Jaśkiewicz & Andrzej S. Nowak, 2020. "Equilibria in Altruistic Economic Growth Models," Dynamic Games and Applications, Springer, vol. 10(1), pages 1-18, March.
    9. Łukasz Balbus & Kevin Reffett & Łukasz Woźny, 2015. "Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers," International Journal of Game Theory, Springer;Game Theory Society, vol. 44(1), pages 83-112, February.
    10. Méder, Zsombor Z. & Flesch, János & Peeters, Ronald, 2017. "Naiveté and sophistication in dynamic inconsistency," Mathematical Social Sciences, Elsevier, vol. 87(C), pages 40-54.
    11. Balbus, Łukasz & Reffett, Kevin & Woźny, Łukasz, 2022. "Time-consistent equilibria in dynamic models with recursive payoffs and behavioral discounting," Journal of Economic Theory, Elsevier, vol. 204(C).
    12. Chatterjee, Satyajit & Eyigungor, Burcu, 2016. "Continuous Markov equilibria with quasi-geometric discounting," Journal of Economic Theory, Elsevier, vol. 163(C), pages 467-494.
    13. A. S. Nowak, 2010. "On a Noncooperative Stochastic Game Played by Internally Cooperating Generations," Journal of Optimization Theory and Applications, Springer, vol. 144(1), pages 88-106, January.
    14. Agah R. Turan, 2019. "Intentional time inconsistency," Theory and Decision, Springer, vol. 86(1), pages 41-64, February.
    15. David S. Ahn & Ryota Iijima & Todd Sarver, 2017. "Naivet� About Temptation and Self-Control: Foundations for Naive Quasi-Hyperbolic Discounting," Cowles Foundation Discussion Papers 2099R, Cowles Foundation for Research in Economics, Yale University, revised Feb 2018.
    16. Jaśkiewicz, Anna & Nowak, Andrzej S., 2022. "A note on topological aspects in dynamic games of resource extraction and economic growth theory," Games and Economic Behavior, Elsevier, vol. 131(C), pages 264-274.
    17. Maria Arvaniti & Chandra K. Krishnamurthy & Anne-Sophie Crépin, 2019. "Time-consistent resource management with regime shifts," CER-ETH Economics working paper series 19/329, CER-ETH - Center of Economic Research (CER-ETH) at ETH Zurich.
    18. Kihlstrom, Richard, 2009. "Risk aversion and the elasticity of substitution in general dynamic portfolio theory: Consistent planning by forward looking, expected utility maximizing investors," Journal of Mathematical Economics, Elsevier, vol. 45(9-10), pages 634-663, September.
    19. Tyson, Christopher J., 2008. "Management of a capital stock by Strotz's naive planner," Journal of Economic Dynamics and Control, Elsevier, vol. 32(7), pages 2214-2239, July.
    20. Thomas Mariotti & Nikolaus Schweizer & Nora Szech & Jonas von Wangenheim, 2023. "Information Nudges and Self-Control," Management Science, INFORMS, vol. 69(4), pages 2182-2197, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:287:y:2020:i:2:d:10.1007_s10479-018-2778-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.