IDEAS home Printed from https://ideas.repec.org/a/spr/joecth/v78y2024i2d10.1007_s00199-024-01565-w.html
   My bibliography  Save this article

Computing perfect stationary equilibria in stochastic games

Author

Listed:
  • Peixuan Li

    (Southeast University)

  • Chuangyin Dang

    (City University of Hong Kong)

  • P. Jean-Jacques Herings

    (Tilburg University)

Abstract

The notion of stationary equilibrium is one of the most crucial solution concepts in stochastic games. However, a stochastic game can have multiple stationary equilibria, some of which may be unstable or counterintuitive. As a refinement of stationary equilibrium, we extend the concept of perfect equilibrium in strategic games to stochastic games and formulate the notion of perfect stationary equilibrium (PeSE). To further promote its applications, we develop a differentiable homotopy method to compute such an equilibrium. We incorporate vanishing logarithmic barrier terms into the payoff functions, thereby constituting a logarithmic-barrier stochastic game. As a result of this barrier game, we attain a continuously differentiable homotopy system. To reduce the number of variables in the homotopy system, we eliminate the Bellman equations through a replacement of variables and derive an equivalent system. We use the equivalent system to establish the existence of a smooth path, which starts from an arbitrary total mixed strategy profile and ends at a PeSE. Extensive numerical experiments, including relevant applications like dynamic oligopoly models and dynamic legislative voting, further affirm the effectiveness and efficiency of the method.

Suggested Citation

  • Peixuan Li & Chuangyin Dang & P. Jean-Jacques Herings, 2024. "Computing perfect stationary equilibria in stochastic games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 78(2), pages 347-387, September.
  • Handle: RePEc:spr:joecth:v:78:y:2024:i:2:d:10.1007_s00199-024-01565-w
    DOI: 10.1007/s00199-024-01565-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00199-024-01565-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00199-024-01565-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Herings, P. Jean-Jacques & Peeters, Ronald J. A. P., 2004. "Stationary equilibria in stochastic games: structure, selection, and computation," Journal of Economic Theory, Elsevier, vol. 118(1), pages 32-60, September.
    2. Doraszelski, Ulrich & Pakes, Ariel, 2007. "A Framework for Applied Dynamic Analysis in IO," Handbook of Industrial Organization, in: Mark Armstrong & Robert Porter (ed.), Handbook of Industrial Organization, edition 1, volume 3, chapter 30, pages 1887-1966, Elsevier.
    3. McKelvey Richard D. & Palfrey Thomas R., 1995. "Quantal Response Equilibria for Normal Form Games," Games and Economic Behavior, Elsevier, vol. 10(1), pages 6-38, July.
    4. Kalyan Chatterjee & Bhaskar Dutia & Debraj Ray & Kunal Sengupta, 2013. "A Noncooperative Theory of Coalitional Bargaining," World Scientific Book Chapters, in: Bargaining in the Shadow of the Market Selected Papers on Bilateral and Multilateral Bargaining, chapter 5, pages 97-111, World Scientific Publishing Co. Pte. Ltd..
    5. P. Herings & Karl Schmedders, 2006. "Computing equilibria in finance economies with incomplete markets and transaction costs," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 27(3), pages 493-512, April.
    6. MERTENS, Jean-François, 1989. "Stable equilibria - a reformulation. Part I. Definition and basic properties," LIDAM Reprints CORE 866, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    7. Bernhard von Stengel & Antoon van den Elzen & Dolf Talman, 2002. "Computing Normal Form Perfect Equilibria for Extensive Two-Person Games," Econometrica, Econometric Society, vol. 70(2), pages 693-715, March.
    8. P. Jean-Jacques Herings, 2000. "Two simple proofs of the feasibility of the linear tracing procedure," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 15(2), pages 485-490.
    9. Maskin, Eric & Tirole, Jean, 2001. "Markov Perfect Equilibrium: I. Observable Actions," Journal of Economic Theory, Elsevier, vol. 100(2), pages 191-219, October.
    10. van den Elzen, A.H. & Talman, A.J.J., 1988. "A procedure for finding Nash equilibria in bi-matrix games," Research Memorandum FEW 334, Tilburg University, School of Economics and Management.
    11. Martin J. Osborne & Ariel Rubinstein, 1994. "A Course in Game Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262650401, April.
    12. Govindan, Srihari & Wilson, Robert, 2009. "Global Newton Method for stochastic games," Journal of Economic Theory, Elsevier, vol. 144(1), pages 414-421, January.
    13. Von Stengel, Bernhard, 2002. "Computing equilibria for two-person games," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 45, pages 1723-1759, Elsevier.
    14. Yin Chen & Chuangyin Dang, 2019. "A Reformulation-Based Simplicial Homotopy Method for Approximating Perfect Equilibria," Computational Economics, Springer;Society for Computational Economics, vol. 54(3), pages 877-891, October.
    15. Peixuan Li & Chuangyin Dang, 2020. "An Arbitrary Starting Tracing Procedure for Computing Subgame Perfect Equilibria," Journal of Optimization Theory and Applications, Springer, vol. 186(2), pages 667-687, August.
    16. AMIR, Rabah, 2001. "Stochastic games in economics and related fields: an overview," LIDAM Discussion Papers CORE 2001060, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    17. P. Jean-Jacques Herings & Ronald J.A.P. Peeters, 2001. "symposium articles: A differentiable homotopy to compute Nash equilibria of n -person games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 18(1), pages 159-185.
    18. Eaves, B. Curtis & Schmedders, Karl, 1999. "General equilibrium models and homotopy methods," Journal of Economic Dynamics and Control, Elsevier, vol. 23(9-10), pages 1249-1279, September.
    19. Eibelshäuser, Steffen & Poensgen, David, 2019. "Markov Quantal Response Equilibrium and a Homotopy Method for Computing and Selecting Markov Perfect Equilibria of Dynamic Stochastic Games," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203603, Verein für Socialpolitik / German Economic Association.
    20. Mihai Manea, 2018. "Intermediation and Resale in Networks," Journal of Political Economy, University of Chicago Press, vol. 126(3), pages 1250-1301.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cao, Yiyin & Dang, Chuangyin, 2022. "A variant of Harsanyi's tracing procedures to select a perfect equilibrium in normal form games," Games and Economic Behavior, Elsevier, vol. 134(C), pages 127-150.
    2. Cao, Yiyin & Dang, Chuangyin & Xiao, Zhongdong, 2022. "A differentiable path-following method to compute subgame perfect equilibria in stationary strategies in robust stochastic games and its applications," European Journal of Operational Research, Elsevier, vol. 298(3), pages 1032-1050.
    3. Chuangyin Dang & P. Jean-Jacques Herings & Peixuan Li, 2022. "An Interior-Point Differentiable Path-Following Method to Compute Stationary Equilibria in Stochastic Games," INFORMS Journal on Computing, INFORMS, vol. 34(3), pages 1403-1418, May.
    4. Yiyin Cao & Chuangyin Dang & Yabin Sun, 2022. "Complementarity Enhanced Nash’s Mappings and Differentiable Homotopy Methods to Select Perfect Equilibria," Journal of Optimization Theory and Applications, Springer, vol. 192(2), pages 533-563, February.
    5. Dang, Chuangyin & Herings, P. Jean-Jacques & Li, Peixuan, 2020. "An Interior-Point Path-Following Method to Compute Stationary Equilibria in Stochastic Games," Research Memorandum 001, Maastricht University, Graduate School of Business and Economics (GSBE).
    6. Yin Chen & Chuangyin Dang, 2019. "A Reformulation-Based Simplicial Homotopy Method for Approximating Perfect Equilibria," Computational Economics, Springer;Society for Computational Economics, vol. 54(3), pages 877-891, October.
    7. P. Herings & Ronald Peeters, 2010. "Homotopy methods to compute equilibria in game theory," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 42(1), pages 119-156, January.
    8. Herings, P. Jean-Jacques & Peeters, Ronald J. A. P., 2004. "Stationary equilibria in stochastic games: structure, selection, and computation," Journal of Economic Theory, Elsevier, vol. 118(1), pages 32-60, September.
    9. Yiyin Cao & Yin Chen & Chuangyin Dang, 2024. "A Variant of the Logistic Quantal Response Equilibrium to Select a Perfect Equilibrium," Journal of Optimization Theory and Applications, Springer, vol. 201(3), pages 1026-1062, June.
    10. Herings, P. Jean-Jacques & Zhan, Yang, 2021. "The computation of pairwise stable networks," Research Memorandum 004, Maastricht University, Graduate School of Business and Economics (GSBE).
    11. Chen, Yin & Dang, Chuangyin, 2020. "An extension of quantal response equilibrium and determination of perfect equilibrium," Games and Economic Behavior, Elsevier, vol. 124(C), pages 659-670.
    12. Yiyin Cao & Yin Chen & Chuangyin Dang, 2024. "A Differentiable Path-Following Method with a Compact Formulation to Compute Proper Equilibria," INFORMS Journal on Computing, INFORMS, vol. 36(2), pages 377-396, March.
    13. Kimmo Berg, 2016. "Elementary Subpaths in Discounted Stochastic Games," Dynamic Games and Applications, Springer, vol. 6(3), pages 304-323, September.
    14. Jean-Jacques Herings, P., 2002. "Universally converging adjustment processes--a unifying approach," Journal of Mathematical Economics, Elsevier, vol. 38(3), pages 341-370, November.
    15. Peixuan Li & Chuangyin Dang, 2020. "An Arbitrary Starting Tracing Procedure for Computing Subgame Perfect Equilibria," Journal of Optimization Theory and Applications, Springer, vol. 186(2), pages 667-687, August.
    16. Bernhard von Stengel & Antoon van den Elzen & Dolf Talman, 2002. "Computing Normal Form Perfect Equilibria for Extensive Two-Person Games," Econometrica, Econometric Society, vol. 70(2), pages 693-715, March.
    17. Govindan, Srihari & Wilson, Robert, 2004. "Computing Nash equilibria by iterated polymatrix approximation," Journal of Economic Dynamics and Control, Elsevier, vol. 28(7), pages 1229-1241, April.
    18. , & ,, 2010. "A theory of regular Markov perfect equilibria in dynamic stochastic games: genericity, stability, and purification," Theoretical Economics, Econometric Society, vol. 5(3), September.
    19. Stuart McDonald & Liam Wagner, 2010. "The Computation of Perfect and Proper Equilibrium for Finite Games via Simulated Annealing," Risk & Uncertainty Working Papers WPR10_1, Risk and Sustainable Management Group, University of Queensland, revised Apr 2010.
    20. Etessami, Kousha, 2021. "The complexity of computing a (quasi-)perfect equilibrium for an n-player extensive form game," Games and Economic Behavior, Elsevier, vol. 125(C), pages 107-140.

    More about this item

    Keywords

    Stochastic games; Stationary equilibria; Perfectness; Logarithmic barrier differentiable homotopy method;
    All these keywords.

    JEL classification:

    • C02 - Mathematical and Quantitative Methods - - General - - - Mathematical Economics
    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joecth:v:78:y:2024:i:2:d:10.1007_s00199-024-01565-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.