IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2411.12725.html
   My bibliography  Save this paper

Reinforcement Learning, Collusion, and the Folk Theorem

Author

Listed:
  • Galit Askenazi-Golan
  • Domenico Mergoni Cecchelli
  • Edward Plumb

Abstract

We explore the behaviour emerging from learning agents repeatedly interacting strategically for a wide range of learning dynamics that includes projected gradient, replicator and log-barrier dynamics. Going beyond the better-understood classes of potential games and zero-sum games, we consider the setting of a general repeated game with finite recall, for different forms of monitoring. We obtain a Folk Theorem-like result and characterise the set of payoff vectors that can be obtained by these dynamics, discovering a wide range of possibilities for the emergence of algorithmic collusion.

Suggested Citation

  • Galit Askenazi-Golan & Domenico Mergoni Cecchelli & Edward Plumb, 2024. "Reinforcement Learning, Collusion, and the Folk Theorem," Papers 2411.12725, arXiv.org.
    • Handle: RePEc:arx:papers:2411.12725
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2411.12725
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Viossat, Yannick, 2007. "The replicator dynamics does not lead to correlated equilibria," Games and Economic Behavior, Elsevier, vol. 59(2), pages 397-407, May.
    2. Kreps, David M & Wilson, Robert, 1982. "Sequential Equilibria," Econometrica, Econometric Society, vol. 50(4), pages 863-894, July.
    3. Michihiro Kandori & Ichiro Obara, 2006. "Efficiency in Repeated Games Revisited: The Role of Private Strategies," Econometrica, Econometric Society, vol. 74(2), pages 499-519, March.
    4. Mailath, George J. & Morris, Stephen, 2002. "Repeated Games with Almost-Public Monitoring," Journal of Economic Theory, Elsevier, vol. 102(1), pages 189-228, January.
    5. Drew Fudenberg & David K. Levine & Satoru Takahashi, 2008. "Perfect public equilibrium when players are patient," World Scientific Book Chapters, in: Drew Fudenberg & David K Levine (ed.), A Long-Run Collaboration On Long-Run Games, chapter 16, pages 345-367, World Scientific Publishing Co. Pte. Ltd..
    6. Gautier, Axel & Ittoo, Ashwin & Van Cleynenbreugel, Pieter, 2020. "AI algorithms, price discrimination and collusion: a technological, economic and legal perspective," LIDAM Reprints CORE 3138, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    7. Swinkels Jeroen M., 1993. "Adjustment Dynamics and Rational Play in Games," Games and Economic Behavior, Elsevier, vol. 5(3), pages 455-484, July.
    8. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    9. repec:dau:papers:123456789/1061 is not listed on IDEAS
    10. Axel Gautier & Ashwin Ittoo & Pieter Cleynenbreugel, 2020. "AI algorithms, price discrimination and collusion: a technological, economic and legal perspective," European Journal of Law and Economics, Springer, vol. 50(3), pages 405-435, December.
    11. Mertikopoulos, Panayotis & Sandholm, William H., 2018. "Riemannian game dynamics," Journal of Economic Theory, Elsevier, vol. 177(C), pages 315-364.
    12. Barlo, Mehmet & Carmona, Guilherme & Sabourian, Hamid, 2016. "Bounded memory Folk Theorem," Journal of Economic Theory, Elsevier, vol. 163(C), pages 728-774.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yamamoto, Yuichi, 2009. "A limit characterization of belief-free equilibrium payoffs in repeated games," Journal of Economic Theory, Elsevier, vol. 144(2), pages 802-824, March.
    2. Martin, Simon & Rasch, Alexander, 2022. "Collusion by algorithm: The role of unobserved actions," DICE Discussion Papers 382, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    3. Daehyeon Park & Doojin Ryu, 2022. "Supply chain ethics and transparency: An agent‐based model approach with Q‐learning agents," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 43(8), pages 3331-3337, December.
    4. Jason D. Hartline & Sheng Long & Chenhao Zhang, 2024. "Regulation of Algorithmic Collusion," Papers 2401.15794, arXiv.org, revised Sep 2024.
    5. Yuichi Yamamoto, 2013. "Individual Learning and Cooperation in Noisy Repeated Games," PIER Working Paper Archive 13-038, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    6. Laura Abrardi & Carlo Cambini & Laura Rondi, 2022. "Artificial intelligence, firms and consumer behavior: A survey," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 969-991, September.
    7. Colombo, Stefano & Filippini, Luigi & Pignataro, Aldo, 2024. "Information sharing, personalized pricing, and collusion," Information Economics and Policy, Elsevier, vol. 66(C).
    8. Dubus, Antoine, 2024. "Behavior-based algorithmic pricing," Information Economics and Policy, Elsevier, vol. 66(C).
    9. Simon Martin & Alexander Rasch, 2022. "Collusion by Algorithm: The Role of Unobserved Actions," CESifo Working Paper Series 9629, CESifo.
    10. Florian Peiseler & Alexander Rasch & Shiva Shekhar, 2022. "Imperfect information, algorithmic price discrimination, and collusion," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(2), pages 516-549, April.
    11. Connor Douglas & Foster Provost & Arun Sundararajan, 2024. "Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete?," Papers 2411.16574, arXiv.org.
    12. Yuichi Yamamoto, 2012. "Individual Learning and Cooperation in Noisy Repeated Games," PIER Working Paper Archive 12-044, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    13. Mailath, George J. & Morris, Stephen, 2002. "Repeated Games with Almost-Public Monitoring," Journal of Economic Theory, Elsevier, vol. 102(1), pages 189-228, January.
    14. Viossat, Yannick, 2008. "Evolutionary dynamics may eliminate all strategies used in correlated equilibrium," Mathematical Social Sciences, Elsevier, vol. 56(1), pages 27-43, July.
    15. Demichelis, Stefano & Ritzberger, Klaus, 2003. "From evolutionary to strategic stability," Journal of Economic Theory, Elsevier, vol. 113(1), pages 51-75, November.
    16. Hörner, Johannes & Takahashi, Satoru, 2016. "How fast do equilibrium payoff sets converge in repeated games?," Journal of Economic Theory, Elsevier, vol. 165(C), pages 332-359.
    17. Richard McLean & Ichiro Obara & Andrew Postlewaite, 2001. "Informational Smallness and Private Monitoring in Repeated Games," PIER Working Paper Archive 05-024, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 20 Jul 2005.
    18. Sugaya, Takuo & Takahashi, Satoru, 2013. "Coordination failure in repeated games with private monitoring," Journal of Economic Theory, Elsevier, vol. 148(5), pages 1891-1928.
    19. Jeffery Ely & Johannes Horner & Wojciech Olszewski, 2004. "Strategic Commitment Versus Flexibility in a Duopoloy with Entry and Exit," Discussion Papers 1381, Northwestern University, Center for Mathematical Studies in Economics and Management Science.
    20. Nick Drydakis, 2022. "Artificial Intelligence and Reduced SMEs’ Business Risks. A Dynamic Capabilities Analysis During the COVID-19 Pandemic," Information Systems Frontiers, Springer, vol. 24(4), pages 1223-1247, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2411.12725. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.