IDEAS home Printed from https://ideas.repec.org/p/tiu/tiutis/5b68f5d7-3209-4a1b-924c-63675a61c23f.html
   My bibliography  Save this paper

Computing Perfect Stationary Equilibria in Stochastic Games

Author

Listed:
  • Li, Peixuan
  • Dang, Chuangyin
  • Herings, P.J.J.

    (Tilburg University, School of Economics and Management)

Abstract

The notion of stationary equilibrium is one of the most crucial solution concepts in stochastic games. However, a stochastic game can have multiple stationary equilibria, some of which may be unstable or counterintuitive. As a refinement of stationary equilibrium, we extend the concept of perfect equilibrium in strategic games to stochastic games and formulate the notion of perfect stationary equilibrium (PeSE). To further promote its applications, we develop a differentiable homotopy method to compute such an equilibrium. We incorporate vanishing logarithmic barrier terms into the payoff functions, thereby constituting a logarithmic-barrier stochastic game. As a result of this barrier game, we attain a continuously differentiable homotopy system. To reduce the number of variables in the homotopy system, we eliminate the Bellman equations through a replacement of variables and derive an equivalent system. We use the equivalent system to establish the existence of a smooth path, which starts from an arbitrary total mixed strategy profile and ends at a PeSE. Extensive numerical experiments, including relevant applications like dynamic oligopoly models and dynamic legislative voting, further affirm the effectiveness and efficiency of the method.
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Li, Peixuan & Dang, Chuangyin & Herings, P.J.J., 2023. "Computing Perfect Stationary Equilibria in Stochastic Games," Other publications TiSEM 5b68f5d7-3209-4a1b-924c-6, Tilburg University, School of Economics and Management.
    • Handle: RePEc:tiu:tiutis:5b68f5d7-3209-4a1b-924c-63675a61c23f
    as

    Download full text from publisher

    File URL: https://pure.uvt.nl/ws/portalfiles/portal/67910419/2023_006.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Armando Gomes & Philippe Jehiel, 2005. "Dynamic Processes of Social and Economic Interactions: On the Persistence of Inefficiencies," Journal of Political Economy, University of Chicago Press, vol. 113(3), pages 626-667, June.
    2. John C. Harsanyi & Reinhard Selten, 1988. "A General Theory of Equilibrium Selection in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262582384, April.
    3. Herings, P. Jean-Jacques & Peeters, Ronald J. A. P., 2004. "Stationary equilibria in stochastic games: structure, selection, and computation," Journal of Economic Theory, Elsevier, vol. 118(1), pages 32-60, September.
    4. Doraszelski, Ulrich & Pakes, Ariel, 2007. "A Framework for Applied Dynamic Analysis in IO," Handbook of Industrial Organization, in: Mark Armstrong & Robert Porter (ed.), Handbook of Industrial Organization, edition 1, volume 3, chapter 30, pages 1887-1966, Elsevier.
    5. Ulrich Doraszelski & Mark Satterthwaite, 2010. "Computable Markov‐perfect industry dynamics," RAND Journal of Economics, RAND Corporation, vol. 41(2), pages 215-243, June.
    6. McKelvey Richard D. & Palfrey Thomas R., 1995. "Quantal Response Equilibria for Normal Form Games," Games and Economic Behavior, Elsevier, vol. 10(1), pages 6-38, July.
    7. Kalyan Chatterjee & Bhaskar Dutia & Debraj Ray & Kunal Sengupta, 2013. "A Noncooperative Theory of Coalitional Bargaining," World Scientific Book Chapters, in: Bargaining in the Shadow of the Market Selected Papers on Bilateral and Multilateral Bargaining, chapter 5, pages 97-111, World Scientific Publishing Co. Pte. Ltd..
    8. P. Herings & Karl Schmedders, 2006. "Computing equilibria in finance economies with incomplete markets and transaction costs," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 27(3), pages 493-512, April.
    9. Richard Ericson & Ariel Pakes, 1995. "Markov-Perfect Industry Dynamics: A Framework for Empirical Work," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 62(1), pages 53-82.
    10. MERTENS, Jean-François, 1989. "Stable equilibria - a reformulation. Part I. Definition and basic properties," LIDAM Reprints CORE 866, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    11. Bernhard von Stengel & Antoon van den Elzen & Dolf Talman, 2002. "Computing Normal Form Perfect Equilibria for Extensive Two-Person Games," Econometrica, Econometric Society, vol. 70(2), pages 693-715, March.
    12. Akira Okada, 2023. "Dynamic bargaining with voluntary participation and externalities," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 75(2), pages 427-452, February.
    13. Wei He, 2022. "Discontinuous stochastic games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 73(4), pages 827-858, June.
    14. P. Jean-Jacques Herings, 2000. "Two simple proofs of the feasibility of the linear tracing procedure," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 15(2), pages 485-490.
    15. Maskin, Eric & Tirole, Jean, 2001. "Markov Perfect Equilibrium: I. Observable Actions," Journal of Economic Theory, Elsevier, vol. 100(2), pages 191-219, October.
    16. van den Elzen, A.H. & Talman, A.J.J., 1988. "A procedure for finding Nash equilibria in bi-matrix games," Research Memorandum FEW 334, Tilburg University, School of Economics and Management.
    17. Jaap H. Abbring & Jeffrey R. Campbell & Jan Tilly & Nan Yang, 2018. "Very Simple Markov‐Perfect Industry Dynamics: Theory," Econometrica, Econometric Society, vol. 86(2), pages 721-735, March.
    18. Martin J. Osborne & Ariel Rubinstein, 1994. "A Course in Game Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262650401, April.
    19. Susanne Goldlücke & Sebastian Kranz, 2018. "Discounted stochastic games with voluntary transfers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 235-263, July.
    20. Govindan, Srihari & Wilson, Robert, 2009. "Global Newton Method for stochastic games," Journal of Economic Theory, Elsevier, vol. 144(1), pages 414-421, January.
    21. Abbring, Jaap & Campbell, J.R. & Tilly, J. & Yang, N., 2018. "Very Simple Markov-Perfect Industry Dynamics (revision of 2017-021) : Empirics," Discussion Paper 2018-040, Tilburg University, Center for Economic Research.
    22. Von Stengel, Bernhard, 2002. "Computing equilibria for two-person games," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 45, pages 1723-1759, Elsevier.
    23. Duggan, John & Kalandrakis, Tasos, 2012. "Dynamic legislative policy making," Journal of Economic Theory, Elsevier, vol. 147(5), pages 1653-1688.
    24. Yin Chen & Chuangyin Dang, 2019. "A Reformulation-Based Simplicial Homotopy Method for Approximating Perfect Equilibria," Computational Economics, Springer;Society for Computational Economics, vol. 54(3), pages 877-891, October.
    25. Peixuan Li & Chuangyin Dang, 2020. "An Arbitrary Starting Tracing Procedure for Computing Subgame Perfect Equilibria," Journal of Optimization Theory and Applications, Springer, vol. 186(2), pages 667-687, August.
    26. AMIR, Rabah, 2001. "Stochastic games in economics and related fields: an overview," LIDAM Discussion Papers CORE 2001060, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    27. P. Jean-Jacques Herings & Ronald J.A.P. Peeters, 2001. "symposium articles: A differentiable homotopy to compute Nash equilibria of n -person games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 18(1), pages 159-185.
    28. Eaves, B. Curtis & Schmedders, Karl, 1999. "General equilibrium models and homotopy methods," Journal of Economic Dynamics and Control, Elsevier, vol. 23(9-10), pages 1249-1279, September.
    29. Eibelshäuser, Steffen & Poensgen, David, 2019. "Markov Quantal Response Equilibrium and a Homotopy Method for Computing and Selecting Markov Perfect Equilibria of Dynamic Stochastic Games," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203603, Verein für Socialpolitik / German Economic Association.
    30. Chuangyin Dang & P. Jean-Jacques Herings & Peixuan Li, 2022. "An Interior-Point Differentiable Path-Following Method to Compute Stationary Equilibria in Stochastic Games," INFORMS Journal on Computing, INFORMS, vol. 34(3), pages 1403-1418, May.
    31. Mihai Manea, 2018. "Intermediation and Resale in Networks," Journal of Political Economy, University of Chicago Press, vol. 126(3), pages 1250-1301.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cao, Yiyin & Dang, Chuangyin, 2022. "A variant of Harsanyi's tracing procedures to select a perfect equilibrium in normal form games," Games and Economic Behavior, Elsevier, vol. 134(C), pages 127-150.
    2. Cao, Yiyin & Dang, Chuangyin & Xiao, Zhongdong, 2022. "A differentiable path-following method to compute subgame perfect equilibria in stationary strategies in robust stochastic games and its applications," European Journal of Operational Research, Elsevier, vol. 298(3), pages 1032-1050.
    3. Chuangyin Dang & P. Jean-Jacques Herings & Peixuan Li, 2022. "An Interior-Point Differentiable Path-Following Method to Compute Stationary Equilibria in Stochastic Games," INFORMS Journal on Computing, INFORMS, vol. 34(3), pages 1403-1418, May.
    4. Dang, Chuangyin & Herings, P. Jean-Jacques & Li, Peixuan, 2020. "An Interior-Point Path-Following Method to Compute Stationary Equilibria in Stochastic Games," Research Memorandum 001, Maastricht University, Graduate School of Business and Economics (GSBE).
    5. Steffen Eibelshäuser & Victor Klockmann & David Poensgen & Alicia von Schenk, 2023. "The Logarithmic Stochastic Tracing Procedure: A Homotopy Method to Compute Stationary Equilibria of Stochastic Games," INFORMS Journal on Computing, INFORMS, vol. 35(6), pages 1511-1526, November.
    6. Yiyin Cao & Chuangyin Dang & Yabin Sun, 2022. "Complementarity Enhanced Nash’s Mappings and Differentiable Homotopy Methods to Select Perfect Equilibria," Journal of Optimization Theory and Applications, Springer, vol. 192(2), pages 533-563, February.
    7. Yin Chen & Chuangyin Dang, 2019. "A Reformulation-Based Simplicial Homotopy Method for Approximating Perfect Equilibria," Computational Economics, Springer;Society for Computational Economics, vol. 54(3), pages 877-891, October.
    8. P. Herings & Ronald Peeters, 2010. "Homotopy methods to compute equilibria in game theory," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 42(1), pages 119-156, January.
    9. Herings, P. Jean-Jacques & Peeters, Ronald J. A. P., 2004. "Stationary equilibria in stochastic games: structure, selection, and computation," Journal of Economic Theory, Elsevier, vol. 118(1), pages 32-60, September.
    10. Duggan, John & Kalandrakis, Tasos, 2012. "Dynamic legislative policy making," Journal of Economic Theory, Elsevier, vol. 147(5), pages 1653-1688.
    11. Yiyin Cao & Yin Chen & Chuangyin Dang, 2024. "A Variant of the Logistic Quantal Response Equilibrium to Select a Perfect Equilibrium," Journal of Optimization Theory and Applications, Springer, vol. 201(3), pages 1026-1062, June.
    12. Herings, P. Jean-Jacques & Zhan, Yang, 2021. "The computation of pairwise stable networks," Research Memorandum 004, Maastricht University, Graduate School of Business and Economics (GSBE).
    13. Chen, Yin & Dang, Chuangyin, 2020. "An extension of quantal response equilibrium and determination of perfect equilibrium," Games and Economic Behavior, Elsevier, vol. 124(C), pages 659-670.
    14. Yiyin Cao & Yin Chen & Chuangyin Dang, 2024. "A Differentiable Path-Following Method with a Compact Formulation to Compute Proper Equilibria," INFORMS Journal on Computing, INFORMS, vol. 36(2), pages 377-396, March.
    15. Kimmo Berg, 2016. "Elementary Subpaths in Discounted Stochastic Games," Dynamic Games and Applications, Springer, vol. 6(3), pages 304-323, September.
    16. Jean-Jacques Herings, P., 2002. "Universally converging adjustment processes--a unifying approach," Journal of Mathematical Economics, Elsevier, vol. 38(3), pages 341-370, November.
    17. Bernhard von Stengel & Antoon van den Elzen & Dolf Talman, 2002. "Computing Normal Form Perfect Equilibria for Extensive Two-Person Games," Econometrica, Econometric Society, vol. 70(2), pages 693-715, March.
    18. Ron N. Borkovsky & Ulrich Doraszelski & Yaroslav Kryukov, 2010. "A User's Guide to Solving Dynamic Stochastic Games Using the Homotopy Method," Operations Research, INFORMS, vol. 58(4-part-2), pages 1116-1132, August.
    19. Jenkins, Mark & Liu, Paul & Matzkin, Rosa L. & McFadden, Daniel L., 2021. "The browser war — Analysis of Markov Perfect Equilibrium in markets with dynamic demand effects," Journal of Econometrics, Elsevier, vol. 222(1), pages 244-260.
    20. , & ,, 2010. "A theory of regular Markov perfect equilibria in dynamic stochastic games: genericity, stability, and purification," Theoretical Economics, Econometric Society, vol. 5(3), September.

    More about this item

    JEL classification:

    • C02 - Mathematical and Quantitative Methods - - General - - - Mathematical Economics
    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tiu:tiutis:5b68f5d7-3209-4a1b-924c-63675a61c23f. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Richard Broekman (email available below). General contact details of provider: https://www.tilburguniversity.edu/about/schools/economics-and-management/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.