IDEAS home Printed from https://ideas.repec.org/a/spr/dyngam/v3y2013i2p128-161.html
   My bibliography  Save this article

On Canonical Forms for Zero-Sum Stochastic Mean Payoff Games

Author

Listed:
  • Endre Boros
  • Khaled Elbassioni
  • Vladimir Gurvich
  • Kazuhisa Makino

Abstract

We consider two-person zero-sum mean payoff undiscounted stochastic games and obtain sufficient conditions for the existence of a saddle point in uniformly optimal stationary strategies. Namely, these conditions enable us to bring the game, by applying potential transformations, to a canonical form in which locally optimal strategies are globally optimal, and hence the value for every initial position and the optimal strategies of both players can be obtained by playing the local game at each state. We show that these conditions hold for the class of additive transition (AT) games, that is, the special case when the transitions at each state can be decomposed into two parts, each controlled completely by one of the two players. An important special case of AT-games form the so-called BWR-games which are played by two players on a directed graph with positions of three types: Black, White and Random. We give an independent proof for the existence of a canonical form in such games, and use this result to derive the existence of a canonical form (and hence, of a saddle point in uniformly optimal stationary strategies) in a wide class of games, which includes stochastic games with perfect information (PI), switching controller (SC) games and additive rewards, additive transition (ARAT) games. Unlike the proof for AT-games, our proof for the BWR-case does not rely on the existence of a saddle point in stationary strategies. We also derive some algorithmic consequences from these our reductions to BWR-games, in terms of solving PI-, and ARAT-games in sub-exponential time. Copyright Springer Science+Business Media New York 2013

Suggested Citation

  • Endre Boros & Khaled Elbassioni & Vladimir Gurvich & Kazuhisa Makino, 2013. "On Canonical Forms for Zero-Sum Stochastic Mean Payoff Games," Dynamic Games and Applications, Springer, vol. 3(2), pages 128-161, June.
  • Handle: RePEc:spr:dyngam:v:3:y:2013:i:2:p:128-161
    DOI: 10.1007/s13235-013-0075-x
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s13235-013-0075-x
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s13235-013-0075-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Awi Federgruen, 1980. "Successive Approximation Methods in Undiscounted Stochastic Games," Operations Research, INFORMS, vol. 28(3-part-ii), pages 794-809, June.
    2. Truman Bewley & Elon Kohlberg, 1978. "On Stochastic Games with Stationary Optimal Strategies," Mathematics of Operations Research, INFORMS, vol. 3(2), pages 104-125, May.
    3. N. N. Pisaruk, 1999. "Mean Cost Cyclical Games," Mathematics of Operations Research, INFORMS, vol. 24(4), pages 817-828, November.
    4. Flesch, J. & Thuijsman, F. & Vrieze, O.J., 2007. "Stochastic games with additive transitions," European Journal of Operational Research, Elsevier, vol. 179(2), pages 483-497, June.
    5. Raghavan, T.E.S. & Tijs, S.H. & Vrieze, O.J., 1985. "On stochastic games with additive reward and transition structure," Other publications TiSEM 28f85a14-9a6e-4ed8-9a4b-a, Tilburg University, School of Economics and Management.
    6. A. J. Hoffman & R. M. Karp, 1966. "On Nonterminating Stochastic Games," Management Science, INFORMS, vol. 12(5), pages 359-370, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Endre Boros & Khaled Elbassioni & Vladimir Gurvich & Kazuhisa Makino, 2017. "A nested family of $$\varvec{k}$$ k -total effective rewards for positional games," International Journal of Game Theory, Springer;Game Theory Society, vol. 46(1), pages 263-293, March.
    2. Endre Boros & Khaled Elbassioni & Vladimir Gurvich & Kazuhisa Makino, 2018. "A Potential Reduction Algorithm for Two-Person Zero-Sum Mean Payoff Stochastic Games," Dynamic Games and Applications, Springer, vol. 8(1), pages 22-41, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Boros, E. & Gurvich, V., 2003. "On Nash-solvability in pure stationary strategies of finite games with perfect information which may have cycles," Mathematical Social Sciences, Elsevier, vol. 46(2), pages 207-241, October.
    2. Endre Boros & Khaled Elbassioni & Vladimir Gurvich & Kazuhisa Makino, 2018. "A Potential Reduction Algorithm for Two-Person Zero-Sum Mean Payoff Stochastic Games," Dynamic Games and Applications, Springer, vol. 8(1), pages 22-41, March.
    3. Anna Jaśkiewicz & Andrzej Nowak, 2015. "On pure stationary almost Markov Nash equilibria in nonzero-sum ARAT stochastic games," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 81(2), pages 169-179, April.
    4. S. K. Neogy & Prasenjit Mondal & Abhijit Gupta & Debasish Ghorui, 2018. "On Solving Mean Payoff Games Using Pivoting Algorithms," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 35(05), pages 1-26, October.
    5. János Flesch & Gijs Schoenmakers & Koos Vrieze, 2009. "Stochastic games on a product state space: the periodic case," International Journal of Game Theory, Springer;Game Theory Society, vol. 38(2), pages 263-289, June.
    6. Flesch, J. & Thuijsman, F. & Vrieze, O.J., 2007. "Stochastic games with additive transitions," European Journal of Operational Research, Elsevier, vol. 179(2), pages 483-497, June.
    7. Jayakumar Subramanian & Amit Sinha & Aditya Mahajan, 2023. "Robustness and Sample Complexity of Model-Based MARL for General-Sum Markov Games," Dynamic Games and Applications, Springer, vol. 13(1), pages 56-88, March.
    8. Eilon Solan, 2018. "The modified stochastic game," International Journal of Game Theory, Springer;Game Theory Society, vol. 47(4), pages 1287-1327, November.
    9. Flesch, J. & Schoenmakers, G.M. & Vrieze, K., 2008. "Stochastic games on a product state space: the periodic case," Research Memorandum 016, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    10. Levy, Yehuda, 2012. "Stochastic games with information lag," Games and Economic Behavior, Elsevier, vol. 74(1), pages 243-256.
    11. Solan, Eilon & Solan, Omri N. & Solan, Ron, 2020. "Jointly controlled lotteries with biased coins," Games and Economic Behavior, Elsevier, vol. 119(C), pages 383-391.
    12. Eilon Solan & Nicolas Vieille, 2010. "Computing uniformly optimal strategies in two-player stochastic games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 42(1), pages 237-253, January.
    13. Doraszelski, Ulrich & Escobar, Juan F., 2019. "Protocol invariance and the timing of decisions in dynamic games," Theoretical Economics, Econometric Society, vol. 14(2), May.
    14. Miquel Oliu-Barton, 2021. "New Algorithms for Solving Zero-Sum Stochastic Games," Mathematics of Operations Research, INFORMS, vol. 46(1), pages 255-267, February.
    15. Johannes Horner & Satoru Takahashi & Nicolas Vieille, 2012. "On the Limit Equilibrium Payoff Set in Repeated and Stochastic Games," Cowles Foundation Discussion Papers 1848, Cowles Foundation for Research in Economics, Yale University.
    16. Flesch, J. & Kuipers, J. & Schoenmakers, G. & Vrieze, K., 2008. "Subgame-perfection in stochastic games with perfect information and recursive payoffs," Research Memorandum 041, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).
    17. Krishnendu Chatterjee & Rupak Majumdar & Thomas Henzinger, 2008. "Stochastic limit-average games are in EXPTIME," International Journal of Game Theory, Springer;Game Theory Society, vol. 37(2), pages 219-234, June.
    18. Hui Zhang & Christian Wernz & Danny R. Hughes, 2018. "A Stochastic Game Analysis of Incentives and Behavioral Barriers in Chronic Disease Management," Service Science, INFORMS, vol. 10(3), pages 302-319, September.
    19. Frederic H. Murphy, 1972. "Row Dropping Procedures for Cutting Plane Algorithms," Discussion Papers 16, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    20. Andrzej Nowak, 2006. "Remarks on sensitive equilibria in stochastic games with additive reward and transition structure," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 64(3), pages 481-494, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:dyngam:v:3:y:2013:i:2:p:128-161. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.