IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0196577.html
   My bibliography  Save this article

Dynamical selection of Nash equilibria using reinforcement learning: Emergence of heterogeneous mixed equilibria

Author

Listed:
  • Robin Nicole
  • Peter Sollich

Abstract

We study the distribution of strategies in a large game that models how agents choose among different double auction markets. We classify the possible mean field Nash equilibria, which include potentially segregated states where an agent population can split into subpopulations adopting different strategies. As the game is aggregative, the actual equilibrium strategy distributions remain undetermined, however. We therefore compare with the results of a reinforcement learning dynamics inspired by Experience-Weighted Attraction (EWA) learning, which at long times leads to Nash equilibria in the appropriate limits of large intensity of choice, low noise (long agent memory) and perfect imputation of missing scores (fictitious play). The learning dynamics breaks the indeterminacy of the Nash equilibria. Non-trivially, depending on how the relevant limits are taken, more than one type of equilibrium can be selected. These include the standard homogeneous mixed and heterogeneous pure states, but also heterogeneous mixed states where different agents play different strategies that are not all pure. The analysis of the reinforcement learning involves Fokker-Planck modeling combined with large deviation methods. The theoretical results are confirmed by multi-agent simulations.

Suggested Citation

  • Robin Nicole & Peter Sollich, 2018. "Dynamical selection of Nash equilibria using reinforcement learning: Emergence of heterogeneous mixed equilibria," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-37, July.
  • Handle: RePEc:plo:pone00:0196577
    DOI: 10.1371/journal.pone.0196577
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0196577
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0196577&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0196577?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Glenn Ellison & Drew Fudenberg, 2003. "Knife-Edge or Plateau: When Do Market Models Tip?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(4), pages 1249-1278.
    2. Carl Chiarella & Giulia Iori, 2002. "A simulation analysis of the microstructure of double auction markets," Quantitative Finance, Taylor & Francis Journals, vol. 2(5), pages 346-353.
    3. Guilherme Carmona, 2003. "Nash and Limit Equilibria of Games with a Continuum of Players," Game Theory and Information 0311004, University Library of Munich, Germany.
    4. Guilherme Carmona, 2004. "Nash equilibria of games with a continuum of players," Nova SBE Working Paper Series wp466, Universidade Nova de Lisboa, Nova School of Business and Economics.
    5. Aleksandra Alorić & Peter Sollich & Peter McBurney & Tobias Galla, 2016. "Emergence of Cooperative Long-Term Market Loyalty in Double Auction Markets," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-26, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robin Nicole & Aleksandra Alori'c & Peter Sollich, 2020. "Fragmentation in trader preferences among multiple markets: Market coexistence versus single market dominance," Papers 2012.04103, arXiv.org, revised Aug 2021.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robin Nicole & Peter Sollich, 2017. "Dynamical selection of Nash equilibria using Experience Weighted Attraction Learning: emergence of heterogeneous mixed equilibria," Papers 1706.09763, arXiv.org.
    2. Daniel Lacker & Kavita Ramanan, 2019. "Rare Nash Equilibria and the Price of Anarchy in Large Static Games," Mathematics of Operations Research, INFORMS, vol. 44(2), pages 400-422, May.
    3. Yang, Jian, 2011. "Asymptotic interpretations for equilibria of nonatomic games," Journal of Mathematical Economics, Elsevier, vol. 47(4-5), pages 491-499.
    4. Aaron Bodoh-Creed & Brent Hickman, 2016. "College Assignment as a Large Contest," Working Papers 2016-27, Becker Friedman Institute for Research In Economics.
    5. Bodoh-Creed, Aaron, 2013. "Efficiency and information aggregation in large uniform-price auctions," Journal of Economic Theory, Elsevier, vol. 148(6), pages 2436-2466.
    6. Bodoh-Creed, Aaron L. & Hickman, Brent R., 2018. "College assignment as a large contest," Journal of Economic Theory, Elsevier, vol. 175(C), pages 88-126.
    7. Wiebke Roß & Jens Weghake, 2018. "Wa(h)re Liebe: Was Online-Dating-Plattformen über zweiseitige Märkte lehren," TUC Working Papers in Economics 0017, Abteilung für Volkswirtschaftslehre, Technische Universität Clausthal (Department of Economics, Technical University Clausthal).
    8. Glenn Ellison & Drew Fudenberg & Markus Möbius, 2004. "Competing Auctions," Journal of the European Economic Association, MIT Press, vol. 2(1), pages 30-66, March.
    9. Jennifer Brown & John Morgan, 2009. "How Much Is a Dollar Worth? Tipping versus Equilibrium Coexistence on Competing Online Auction Sites," Journal of Political Economy, University of Chicago Press, vol. 117(4), pages 668-700, August.
    10. Renato Gomes & Alessandro Pavan, 2013. "Cross-Subsidization and Matching Design," Discussion Papers 1559, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    11. Robin Nicole & Aleksandra Alori'c & Peter Sollich, 2020. "Fragmentation in trader preferences among multiple markets: Market coexistence versus single market dominance," Papers 2012.04103, arXiv.org, revised Aug 2021.
    12. Carlos Alós-Ferrer & Georg Kirchsteiger & Markus Walzl, 2010. "On the Evolution of Market Institutions: The Platform Design Paradox," Economic Journal, Royal Economic Society, vol. 120(543), pages 215-243, March.
    13. Scalas, Enrico & Kaizoji, Taisei & Kirchler, Michael & Huber, Jürgen & Tedeschi, Alessandra, 2006. "Waiting times between orders and trades in double-auction markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 366(C), pages 463-471.
    14. Simon Loertscher, 2005. "Market making oligopoly," Diskussionsschriften dp0512, Universitaet Bern, Departement Volkswirtschaft.
    15. Attila Ambrus & Emilio Calvano & Markus Reisinger, 2016. "Either or Both Competition: A "Two-Sided" Theory of Advertising with Overlapping Viewerships," American Economic Journal: Microeconomics, American Economic Association, vol. 8(3), pages 189-222, August.
    16. Yeh, Chia-Hsuan & Yang, Chun-Yi, 2010. "Examining the effectiveness of price limits in an artificial stock market," Journal of Economic Dynamics and Control, Elsevier, vol. 34(10), pages 2089-2108, October.
    17. Guilherme Carmona, 2004. "On the existence of pure strategy nash equilibria in large games," Nova SBE Working Paper Series wp465, Universidade Nova de Lisboa, Nova School of Business and Economics.
    18. Chiarella, Carl & Ladley, Daniel, 2016. "Chasing trends at the micro-level: The effect of technical trading on order book dynamics," Journal of Banking & Finance, Elsevier, vol. 72(S), pages 119-131.
    19. Alberto Ciacci & Takumi Sueshige & Hideki Takayasu & Kim Christensen & Misako Takayasu, 2020. "The microscopic relationships between triangular arbitrage and cross-currency correlations in a simple agent based model of foreign exchange markets," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-19, June.
    20. Lemoine, Derek M. & Traeger, Christian P., 2010. "Tipping Points and Ambiguity in the Economics of Climate Change," CUDARE Working Papers 98127, University of California, Berkeley, Department of Agricultural and Resource Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0196577. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.