IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1910.00193.html
   My bibliography  Save this paper

Parallel Algorithm for Approximating Nash Equilibrium in Multiplayer Stochastic Games with Application to Naval Strategic Planning

Author

Listed:
  • Sam Ganzfried
  • Conner Laughlin
  • Charles Morefield

Abstract

Many real-world domains contain multiple agents behaving strategically with probabilistic transitions and uncertain (potentially infinite) duration. Such settings can be modeled as stochastic games. While algorithms have been developed for solving (i.e., computing a game-theoretic solution concept such as Nash equilibrium) two-player zero-sum stochastic games, research on algorithms for non-zero-sum and multiplayer stochastic games is limited. We present a new algorithm for these settings, which constitutes the first parallel algorithm for multiplayer stochastic games. We present experimental results on a 4-player stochastic game motivated by a naval strategic planning scenario, showing that our algorithm is able to quickly compute strategies constituting Nash equilibrium up to a very small degree of approximation error.

Suggested Citation

  • Sam Ganzfried & Conner Laughlin & Charles Morefield, 2019. "Parallel Algorithm for Approximating Nash Equilibrium in Multiplayer Stochastic Games with Application to Naval Strategic Planning," Papers 1910.00193, arXiv.org, revised Mar 2020.
    • Handle: RePEc:arx:papers:1910.00193
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1910.00193
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fudenberg, Drew & Levine, David, 1998. "Learning in games," European Economic Review, Elsevier, vol. 42(3-5), pages 631-639, May.
    2. Porter, Ryan & Nudelman, Eugene & Shoham, Yoav, 2008. "Simple search methods for finding a Nash equilibrium," Games and Economic Behavior, Elsevier, vol. 63(2), pages 642-662, July.
    3. Govindan, Srihari & Wilson, Robert, 2003. "A global Newton method to compute Nash equilibria," Journal of Economic Theory, Elsevier, vol. 110(1), pages 65-86, May.
    4. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945, April.
    5. Nicolas Vieille, 2000. "Two-player stochastic games I: A reduction," Post-Print hal-00481401, HAL.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sam Ganzfried, 2021. "Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto," Games, MDPI, vol. 12(2), pages 1-11, June.
    2. Sam Ganzfried, 2022. "Fictitious Play with Maximin Initialization," Papers 2203.10774, arXiv.org, revised Nov 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joseph Y. Halpern, 2007. "Computer Science and Game Theory: A Brief Survey," Papers cs/0703148, arXiv.org.
    2. Galbiati, Marco & Soramäki, Kimmo, 2011. "An agent-based model of payment systems," Journal of Economic Dynamics and Control, Elsevier, vol. 35(6), pages 859-875, June.
    3. Schipper, Burkhard C., 2021. "Discovery and equilibrium in games with unawareness," Journal of Economic Theory, Elsevier, vol. 198(C).
    4. Mathieu Faure & Gregory Roth, 2010. "Stochastic Approximations of Set-Valued Dynamical Systems: Convergence with Positive Probability to an Attractor," Mathematics of Operations Research, INFORMS, vol. 35(3), pages 624-640, August.
    5. Ianni, A., 2002. "Reinforcement learning and the power law of practice: some analytical results," Discussion Paper Series In Economics And Econometrics 203, Economics Division, School of Social Sciences, University of Southampton.
    6. ,, 2011. "Manipulative auction design," Theoretical Economics, Econometric Society, vol. 6(2), May.
    7. Christian Ewerhart, 2020. "Ordinal potentials in smooth games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 70(4), pages 1069-1100, November.
    8. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.
    9. Saori Iwanaga & Akira Namatame, 2015. "Hub Agents Determine Collective Behavior," New Mathematics and Natural Computation (NMNC), World Scientific Publishing Co. Pte. Ltd., vol. 11(02), pages 165-181.
    10. Erhao Xie, 2019. "Monetary Payoff and Utility Function in Adaptive Learning Models," Staff Working Papers 19-50, Bank of Canada.
    11. Jacob W. Crandall & Mayada Oudah & Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael A. Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," TSE Working Papers 17-806, Toulouse School of Economics (TSE).
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," IAST Working Papers 17-68, Institute for Advanced Study in Toulouse (IAST).
      • Jacob Crandall & Mayada Oudah & Fatimah Ishowo-Oloko Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Post-Print hal-01897802, HAL.
    12. Dieter Balkenborg & Rosemarie Nagel, 2016. "An Experiment on Forward vs. Backward Induction: How Fairness and Level k Reasoning Matter," German Economic Review, Verein für Socialpolitik, vol. 17(3), pages 378-408, August.
    13. P. Giovani Palafox-Alcantar & Dexter V. L. Hunt & Chris D. F. Rogers, 2020. "A Hybrid Methodology to Study Stakeholder Cooperation in Circular Economy Waste Management of Cities," Energies, MDPI, vol. 13(7), pages 1-30, April.
    14. B Kelsey Jack, 2009. "Auctioning Conservation Contracts in Indonesia - Participant Learning in Multiple Trial Rounds," CID Working Papers 35, Center for International Development at Harvard University.
    15. Waters, George A., 2009. "Chaos in the cobweb model with a new learning dynamic," Journal of Economic Dynamics and Control, Elsevier, vol. 33(6), pages 1201-1216, June.
    16. William L. Cooper & Tito Homem-de-Mello & Anton J. Kleywegt, 2015. "Learning and Pricing with Models That Do Not Explicitly Incorporate Competition," Operations Research, INFORMS, vol. 63(1), pages 86-103, February.
    17. Carlos Alós-Ferrer & Georg Kirchsteiger & Markus Walzl, 2010. "On the Evolution of Market Institutions: The Platform Design Paradox," Economic Journal, Royal Economic Society, vol. 120(543), pages 215-243, March.
    18. Siegfried Berninghaus & Werner Güth & M. Vittoria Levati & Jianying Qiu, 2006. "Satisficing in sales competition: experimental evidence," Papers on Strategic Interaction 2006-32, Max Planck Institute of Economics, Strategic Interaction Group.
    19. Cho, In-Koo, 2005. "Introduction to learning and bounded rationality," Journal of Economic Theory, Elsevier, vol. 124(2), pages 127-128, October.
    20. Ball, Richard, 2017. "Violations of monotonicity in evolutionary models with sample-based beliefs," Economics Letters, Elsevier, vol. 152(C), pages 100-104.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1910.00193. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.