IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v188y2021i3d10.1007_s10957-021-01812-3.html
   My bibliography  Save this article

A Proximal/Gradient Approach for Computing the Nash Equilibrium in Controllable Markov Games

Author

Listed:
  • Julio B. Clempner

    (National Polytechnic Institute)

Abstract

This paper proposes a new algorithm for computing the Nash equilibrium based on an iterative approach of both the proximal and the gradient method for homogeneous, finite, ergodic and controllable Markov chains. We conceptualize the problem as a poly-linear programming problem. Then, we regularize the poly-linear functional employing a regularization approach over the Lagrange functional for ensuring the method to converge to some of the Nash equilibria of the game. This paper presents two main contributions: The first theoretical result is the proposed iterative approach, which employs both the proximal and the gradient method for computing the Nash equilibria in Markov games. The method transforms the game theory problem in a system of equations, in which each equation itself is an independent optimization problem for which the necessary condition of a minimum is computed employing a nonlinear programming solver. The iterated approach provides a quick rate of convergence to the Nash equilibrium point. The second computational contribution focuses on the analysis of the convergence of the proposed method and computes the rate of convergence of the step-size parameter. These results are interesting within the context of computational and algorithmic game theory. A numerical example illustrates the proposed approach.

Suggested Citation

  • Julio B. Clempner, 2021. "A Proximal/Gradient Approach for Computing the Nash Equilibrium in Controllable Markov Games," Journal of Optimization Theory and Applications, Springer, vol. 188(3), pages 847-862, March.
    • Handle: RePEc:spr:joptap:v:188:y:2021:i:3:d:10.1007_s10957-021-01812-3
    DOI: 10.1007/s10957-021-01812-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-021-01812-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-021-01812-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Steven Gabriel & Sauleh Siddiqui & Antonio Conejo & Carlos Ruiz, 2013. "Solving Discretely-Constrained Nash–Cournot Games with an Application to Power Markets," Networks and Spatial Economics, Springer, vol. 13(3), pages 307-326, September.
    2. Axel Dreves & Christian Kanzow & Oliver Stein, 2012. "Nonsmooth optimization reformulations of player convex generalized Nash equilibrium problems," Journal of Global Optimization, Springer, vol. 53(4), pages 587-614, August.
    3. Koichi Nabetani & Paul Tseng & Masao Fukushima, 2011. "Parametrized variational inequality approaches to generalized Nash equilibrium problems with shared constraints," Computational Optimization and Applications, Springer, vol. 48(3), pages 423-452, April.
    4. Julio B. Clempner, 2018. "Computing multiobjective Markov chains handled by the extraproximal method," Annals of Operations Research, Springer, vol. 271(2), pages 469-486, December.
    5. Julio B. Clempner & Alexander S. Poznyak, 2020. "Finding the Strong Nash Equilibrium: Computation, Existence and Characterization for Markov Games," Journal of Optimization Theory and Applications, Springer, vol. 186(3), pages 1029-1052, September.
    6. Julio B. Clempner, 2015. "Setting Cournot Versus Lyapunov Games Stability Conditions and Equilibrium Point Properties," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 17(04), pages 1-10.
    7. Julio B. CLEMPNER & Alexander S. POZNYAK, 2016. "Analyzing An Optimistic Attitude For The Leader Firm In Duopoly Models: A Strong Stackelberg Equilibrium Based On A Lyapunov Game Theory Approach," ECONOMIC COMPUTATION AND ECONOMIC CYBERNETICS STUDIES AND RESEARCH, Faculty of Economic Cybernetics, Statistics and Informatics, vol. 50(4), pages 41-60.
    8. Clempner, Julio B. & Poznyak, Alexander S., 2015. "Computing the strong Nash equilibrium for Markov chains games," Applied Mathematics and Computation, Elsevier, vol. 265(C), pages 911-927.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Julio B. Clempner, 2023. "A Dynamic Mechanism Design for Controllable and Ergodic Markov Games," Computational Economics, Springer;Society for Computational Economics, vol. 61(3), pages 1151-1171, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simone Sagratella, 2017. "Algorithms for generalized potential games with mixed-integer variables," Computational Optimization and Applications, Springer, vol. 68(3), pages 689-717, December.
    2. Simone Sagratella, 2017. "Computing equilibria of Cournot oligopoly models with mixed-integer quantities," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 86(3), pages 549-565, December.
    3. Axel Dreves, 2018. "How to Select a Solution in Generalized Nash Equilibrium Problems," Journal of Optimization Theory and Applications, Springer, vol. 178(3), pages 973-997, September.
    4. Axel Dreves, 2014. "Finding all solutions of affine generalized Nash equilibrium problems with one-dimensional strategy sets," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 80(2), pages 139-159, October.
    5. Axel Dreves, 2017. "Computing all solutions of linear generalized Nash equilibrium problems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 85(2), pages 207-221, April.
    6. Julio B. Clempner, 2018. "Strategic Manipulation Approach for Solving Negotiated Transfer Pricing Problem," Journal of Optimization Theory and Applications, Springer, vol. 178(1), pages 304-316, July.
    7. Oliver Stein & Nathan Sudermann-Merx, 2016. "The Cone Condition and Nonsmoothness in Linear Generalized Nash Games," Journal of Optimization Theory and Applications, Springer, vol. 170(2), pages 687-709, August.
    8. Dávid Csercsik & László Á. Kóczy, 2017. "Efficiency and Stability in Electrical Power Transmission Networks: a Partition Function Form Approach," Networks and Spatial Economics, Springer, vol. 17(4), pages 1161-1184, December.
    9. Julio B. Clempner, 2023. "A Dynamic Mechanism Design for Controllable and Ergodic Markov Games," Computational Economics, Springer;Society for Computational Economics, vol. 61(3), pages 1151-1171, March.
    10. Nadja Harms & Tim Hoheisel & Christian Kanzow, 2015. "On a Smooth Dual Gap Function for a Class of Player Convex Generalized Nash Equilibrium Problems," Journal of Optimization Theory and Applications, Springer, vol. 166(2), pages 659-685, August.
    11. Friedrich Kunz & Alexander Zerrahn, 2013. "The Benefit of Coordinating Congestion Management in Germany," Discussion Papers of DIW Berlin 1298, DIW Berlin, German Institute for Economic Research.
    12. Hassan Shavandi & Mehrdad Pirnia & J. David Fuller, 2018. "Extended opportunity cost model to find near equilibrium electricity prices under non-convexities," Papers 1809.09734, arXiv.org.
    13. Letícia Becher & Damián Fernández & Alberto Ramos, 2023. "A trust-region LP-Newton method for constrained nonsmooth equations under Hölder metric subregularity," Computational Optimization and Applications, Springer, vol. 86(2), pages 711-743, November.
    14. Wen Zhou & Nikita Koptyug & Shutao Ye & Yifan Jia & Xiaolong Lu, 2016. "An Extended N-Player Network Game and Simulation of Four Investment Strategies on a Complex Innovation Network," PLOS ONE, Public Library of Science, vol. 11(1), pages 1-18, January.
    15. Jiang, Zhoutong & Lei, Chao & Ouyang, Yanfeng, 2020. "Optimal investment and management of shared bikes in a competitive market," Transportation Research Part B: Methodological, Elsevier, vol. 135(C), pages 143-155.
    16. Alexey Izmailov & Mikhail Solodov, 2014. "On error bounds and Newton-type methods for generalized Nash equilibrium problems," Computational Optimization and Applications, Springer, vol. 59(1), pages 201-218, October.
    17. Dávid Csercsik, 2016. "Competition and Cooperation in a Bidding Model of Electrical Energy Trade," Networks and Spatial Economics, Springer, vol. 16(4), pages 1043-1073, December.
    18. Migot, Tangi & Cojocaru, Monica-G., 2020. "A parametrized variational inequality approach to track the solution set of a generalized nash equilibrium problem," European Journal of Operational Research, Elsevier, vol. 283(3), pages 1136-1147.
    19. Axel Dreves & Christian Kanzow & Oliver Stein, 2012. "Nonsmooth optimization reformulations of player convex generalized Nash equilibrium problems," Journal of Global Optimization, Springer, vol. 53(4), pages 587-614, August.
    20. Le Cadre, Hélène & Mezghani, Ilyès & Papavasiliou, Anthony, 2019. "A game-theoretic analysis of transmission-distribution system operator coordination," European Journal of Operational Research, Elsevier, vol. 274(1), pages 317-339.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:188:y:2021:i:3:d:10.1007_s10957-021-01812-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.