IDEAS home Printed from https://ideas.repec.org/p/bie/wpaper/475.html
   My bibliography  Save this paper

A limit theorem for Markov decision processes

Author

Listed:
  • Staudigl, Mathias

    (Center for Mathematical Economics, Bielefeld University)

Abstract

In this paper we prove a deterministic approximation theorem for a sequence of Markov decision processes with finitely many actions and general state spaces as they appear frequently in economics, game theory and operations research. Using viscosity solution methods no a-priori differentiabililty assumptions are imposed on the value function. Applications for this result can be found in large deviation theory, and some simple economic problems.

Suggested Citation

  • Staudigl, Mathias, 2014. "A limit theorem for Markov decision processes," Center for Mathematical Economics Working Papers 475, Center for Mathematical Economics, Bielefeld University.
    • Handle: RePEc:bie:wpaper:475
    as

    Download full text from publisher

    File URL: https://pub.uni-bielefeld.de/download/2674039/2901847
    File Function: First Version, 2013
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Johannes Hörner & Takuo Sugaya & Satoru Takahashi & Nicolas Vieille, 2011. "Recursive Methods in Discounted Stochastic Games: An Algorithm for δ→ 1 and a Folk Theorem," Econometrica, Econometric Society, vol. 79(4), pages 1277-1318, July.
    2. Yuliy Sannikov & Andrzej Skrzypacz, 2010. "The Role of Information in Repeated Games With Frequent Actions," Econometrica, Econometric Society, vol. 78(3), pages 847-882, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fudenberg, Drew & Ishii, Yuhta & Kominers, Scott Duke, 2014. "Delayed-response strategies in repeated games with observation lags," Journal of Economic Theory, Elsevier, vol. 150(C), pages 487-514.
    2. Hörner, Johannes & Takahashi, Satoru, 2016. "How fast do equilibrium payoff sets converge in repeated games?," Journal of Economic Theory, Elsevier, vol. 165(C), pages 332-359.
    3. Daehyun Kim & Ichiro Obara, 2023. "Asymptotic Value of Monitoring Structures in Stochastic Games," Papers 2308.09211, arXiv.org, revised Jul 2024.
    4. Aislinn Bohren, 2016. "Using Persistence to Generate Incentives in a Dynamic Moral Hazard Problem," PIER Working Paper Archive 16-024, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 15 Oct 2016.
    5. Johannes H�rner & Satoru Takahashi & Nicolas Vieille, 2012. "On the Limit Equilibrium Payoff Set in Repeated and Stochastic Games," Working Papers 1397, Princeton University, Department of Economics, Econometric Research Program..
    6. Kimmo Berg & Gijs Schoenmakers, 2017. "Construction of Subgame-Perfect Mixed-Strategy Equilibria in Repeated Games," Games, MDPI, vol. 8(4), pages 1-14, November.
    7. , & ,, 2015. "A folk theorem for stochastic games with infrequent state changes," Theoretical Economics, Econometric Society, vol. 10(1), January.
    8. Johannes Hörner & Nicolas Klein & Sven Rady, 2022. "Overcoming Free-Riding in Bandit Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(4), pages 1948-1992.
    9. Lehrer, Ehud & Solan, Eilon, 2018. "High frequency repeated games with costly monitoring," Theoretical Economics, Econometric Society, vol. 13(1), January.
    10. Kranz, Sebastian, 2013. "Relational Contracting, Repeated Negotiations, and Hold-Up," VfS Annual Conference 2013 (Duesseldorf): Competition Policy and Regulation in a Global Economic Order 80047, Verein für Socialpolitik / German Economic Association.
    11. Escobar, Juan F. & Llanes, Gastón, 2018. "Cooperation dynamics in repeated games of adverse selection," Journal of Economic Theory, Elsevier, vol. 176(C), pages 408-443.
    12. Drew Fudenberg & David K. Levine, 2008. "Continuous time limits of repeated games with imperfect public monitoring," World Scientific Book Chapters, in: Drew Fudenberg & David K Levine (ed.), A Long-Run Collaboration On Long-Run Games, chapter 17, pages 369-388, World Scientific Publishing Co. Pte. Ltd..
    13. Renault, Jérôme & Solan, Eilon & Vieille, Nicolas, 2013. "Dynamic sender–receiver games," Journal of Economic Theory, Elsevier, vol. 148(2), pages 502-534.
    14. Susanne Goldlücke & Sebastian Kranz, 2018. "Discounted stochastic games with voluntary transfers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 235-263, July.
    15. Eilon Solan, 2018. "The modified stochastic game," International Journal of Game Theory, Springer;Game Theory Society, vol. 47(4), pages 1287-1327, November.
    16. Guéron, Yves, 2015. "Failure of gradualism under imperfect monitoring," Journal of Economic Theory, Elsevier, vol. 157(C), pages 128-145.
    17. Sven Rady & Nicolas Klein & Johannes Horner, 2013. "Strongly Symmetric Equilibria in Bandit Games," 2013 Meeting Papers 1107, Society for Economic Dynamics.
    18. Fudenberg, Drew & Olszewski, Wojciech, 2011. "Repeated games with asynchronous monitoring of an imperfect signal," Games and Economic Behavior, Elsevier, vol. 72(1), pages 86-99, May.
    19. Du, Chuang, 2012. "Solving payoff sets of perfect public equilibria: an example," MPRA Paper 38622, University Library of Munich, Germany.
    20. Pierre Cardaliaguet & Catherine Rainer & Dinah Rosenberg & Nicolas Vieille, 2016. "Markov Games with Frequent Actions and Incomplete Information—The Limit Case," Mathematics of Operations Research, INFORMS, vol. 41(1), pages 49-71, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bie:wpaper:475. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bettina Weingarten (email available below). General contact details of provider: https://edirc.repec.org/data/imbiede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.