IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v355y2005i1p119-130.html
   My bibliography  Save this article

Convergence of reinforcement learning to Nash equilibrium: A search-market experiment

Author

Listed:
  • Darmon, Eric
  • Waldeck, Roger

Abstract

Since the introduction of Reinforcement Learning (RL) in Game Theory, a growing literature is concerned with the theoretical convergence of RL-driven outcomes towards Nash equilibrium. In this paper, we apply this issue to a search-theoretic framework (posted-price market) where sellers are confronted with a population of imperfectly informed buyers and take one decision per period (posted prices) with no direct interactions between sellers. We focus on three different scenarios with varying buyers’ characteristics. For each of these scenarios, we quantitatively and qualitatively test whether the learned variable (price strategy) converges to the Nash equilibrium. We also study the impact of the temperature parameter (defining the exploitation/exploration trade off) on these results.

Suggested Citation

  • Darmon, Eric & Waldeck, Roger, 2005. "Convergence of reinforcement learning to Nash equilibrium: A search-market experiment," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 355(1), pages 119-130.
    • Handle: RePEc:eee:phsmap:v:355:y:2005:i:1:p:119-130
    DOI: 10.1016/j.physa.2005.02.074
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437105002839
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2005.02.074?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    2. Kirman, Alan P. & Vriend, Nicolaas J., 2001. "Evolving market structure: An ACE model of price dispersion and loyalty," Journal of Economic Dynamics and Control, Elsevier, vol. 25(3-4), pages 459-502, March.
    3. Martin Posch, 1997. "Cycling in a stochastic learning algorithm for normal form games," Journal of Evolutionary Economics, Springer, vol. 7(2), pages 193-207.
    4. Erik Brynjolfsson & Michael D. Smith, 2000. "Frictionless Commerce? A Comparison of Internet and Conventional Retailers," Management Science, INFORMS, vol. 46(4), pages 563-585, April.
    5. Bell, Ann Maria, 2001. "Reinforcement Learning Rules in a Repeated Game," Computational Economics, Springer;Society for Computational Economics, vol. 18(1), pages 89-110, August.
    6. Brenner, Thomas, 2002. "A Behavioural Learning Approach to the Dynamics of Prices," Computational Economics, Springer;Society for Computational Economics, vol. 19(1), pages 67-94, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fogale, Alberto & Pellizzari, Paolo & Warglien, Massimo, 2007. "Learning and equilibrium selection in a coordination game with heterogeneous agents," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 380(C), pages 519-527.
    2. Roger Waldeck & Eric Darmon, 2006. "Can boundedly rational sellers learn to play Nash?," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 1(2), pages 147-169, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roger Waldeck & Eric Darmon, 2006. "Can boundedly rational sellers learn to play Nash?," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 1(2), pages 147-169, November.
    2. Waltman, Ludo & Kaymak, Uzay, 2008. "Q-learning agents in a Cournot oligopoly model," Journal of Economic Dynamics and Control, Elsevier, vol. 32(10), pages 3275-3293, October.
    3. Moulet, Sonia & Rouchier, Juliette, 2008. "The influence of seller learning and time constraints on sequential bargaining in an artificial perishable goods market," Journal of Economic Dynamics and Control, Elsevier, vol. 32(7), pages 2322-2348, July.
    4. Lamieri, Marco & Bertacchini, Enrico, 2006. "What if Hayek goes shopping in the bazaar?," MPRA Paper 367, University Library of Munich, Germany, revised 21 Jun 2006.
    5. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.
    6. Alan Kirman & Sonia Moulet, 2008. "Impact de l'organisation du marché: Comparaison de la négociation de gré à gré et des enchères descendantes," Working Papers halshs-00349034, HAL.
    7. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    8. Hopkins, Ed, 2007. "Adaptive learning models of consumer behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 64(3-4), pages 348-368.
    9. Stephan Schuster, 2012. "BRA: An Algorithm for Simulating Bounded Rational Agents," Computational Economics, Springer;Society for Computational Economics, vol. 39(1), pages 51-69, January.
    10. Ed Hopkins & Robert M. Seymour, 2002. "The Stability of Price Dispersion under Seller and Consumer Learning," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 43(4), pages 1157-1190, November.
    11. Juliette Rouchier, 2013. "The Interest of Having Loyal Buyers in a Perishable Market," Computational Economics, Springer;Society for Computational Economics, vol. 41(2), pages 151-170, February.
    12. Alanyali, Murat, 2010. "A note on adjusted replicator dynamics in iterated games," Journal of Mathematical Economics, Elsevier, vol. 46(1), pages 86-98, January.
    13. Leigh Tesfatsion, 2002. "Agent-Based Computational Economics," Computational Economics 0203001, University Library of Munich, Germany, revised 15 Aug 2002.
    14. Mengel, Friederike, 2012. "Learning across games," Games and Economic Behavior, Elsevier, vol. 74(2), pages 601-619.
    15. R. Cowan & N. Jonard & J. -B. Zimmermann, 2007. "Evolving networks of inventors," Springer Books, in: Uwe Cantner & Franco Malerba (ed.), Innovation, Industrial Dynamics and Structural Transformation, pages 129-148, Springer.
    16. Hopkins, Ed & Posch, Martin, 2005. "Attainability of boundary points under reinforcement learning," Games and Economic Behavior, Elsevier, vol. 53(1), pages 110-125, October.
    17. Laslier, Jean-Francois & Topol, Richard & Walliser, Bernard, 2001. "A Behavioral Learning Process in Games," Games and Economic Behavior, Elsevier, vol. 37(2), pages 340-366, November.
    18. Mario Bravo & Mathieu Faure, 2013. "Reinforcement Learning with Restrictions on the Action Set," AMSE Working Papers 1335, Aix-Marseille School of Economics, France, revised 01 Jul 2013.
    19. Eymann, Torsten & Streitberger, Werner & Reinicke, Michael & Freitag, Felix & Chacin, Pablo & Chao, Isaac & Schnizler, Björn & Veit, Daniel, 2007. "Preliminary specification and design documentation for software components to achieve catallaxy in computational systems," Bayreuth Reports on Information Systems Management 2, University of Bayreuth, Chair of Information Systems Management.
    20. Sheri Markose, 2006. "Developments in experimental and agent-based computational economics (ACE): overview," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 1(2), pages 119-127, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:355:y:2005:i:1:p:119-130. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.