IDEAS home Printed from https://ideas.repec.org/p/aep/anales/4587.html
   My bibliography  Save this paper

Q-Learning algorithms in a Hotelling model

Author

Listed:
  • Lucila Porto

Abstract

What if Q-Learning algorithms set not only prices but also the degree of differentiation between them? In this paper, I tackle this question by analyzing the competition between two Q-Learning algorithms in a Hotelling setting. I find that most of the simulations converge to a Nash Equilibrium where the algorithms are playing non-competitive strategies. In most simulations, they optimally learn not to differentiate each other and to set a collusive price. An underlying deviation and punishment scheme sustains this implicit agreement. The results are robust to the enlargement of the action space and the introduction of relocalization costs.

Suggested Citation

  • Lucila Porto, 2022. "Q-Learning algorithms in a Hotelling model," Asociación Argentina de Economía Política: Working Papers 4587, Asociación Argentina de Economía Política.
    • Handle: RePEc:aep:anales:4587
    as

    Download full text from publisher

    File URL: https://aaep.org.ar/works/works2022/4587.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Waltman, Ludo & Kaymak, Uzay, 2008. "Q-learning agents in a Cournot oligopoly model," Journal of Economic Dynamics and Control, Elsevier, vol. 32(10), pages 3275-3293, October.
    2. Calvano, Emilio & Calzolari, Giacomo & Denicoló, Vincenzo & Pastorello, Sergio, 2021. "Algorithmic collusion with imperfect monitoring," International Journal of Industrial Organization, Elsevier, vol. 79(C).
    3. Zach Y. Brown & Alexander MacKay, 2023. "Competition in Pricing Algorithms," American Economic Journal: Microeconomics, American Economic Association, vol. 15(2), pages 109-156, May.
    4. John Asker & Chaim Fershtman & Ariel Pakes, 2021. "Artificial Intelligence and Pricing: The Impact of Algorithm Design," NBER Working Papers 28535, National Bureau of Economic Research, Inc.
    5. John Asker & Chaim Fershtman & Ariel Pakes, 2022. "Artificial Intelligence, Algorithm Design, and Pricing," AEA Papers and Proceedings, American Economic Association, vol. 112, pages 452-456, May.
    6. Joseph E Harrington, 2018. "Developing Competition Law For Collusion By Autonomous Artificial Agents," Journal of Competition Law and Economics, Oxford University Press, vol. 14(3), pages 331-363.
    7. Marcel Wieting & Geza Sapi, 2021. "Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing in E-Commerce," Working Papers 21-06, NET Institute.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dolgopolov, Arthur, 2024. "Reinforcement learning in a prisoner's dilemma," Games and Economic Behavior, Elsevier, vol. 144(C), pages 84-103.
    2. Werner, Tobias, 2021. "Algorithmic and human collusion," DICE Discussion Papers 372, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    3. Simon Martin & Alexander Rasch, 2022. "Collusion by Algorithm: The Role of Unobserved Actions," CESifo Working Paper Series 9629, CESifo.
    4. Fourberg, Niklas & Marques Magalhaes, Katrin & Wiewiorra, Lukas, 2023. "They Are Among Us: Pricing Behavior of Algorithms in the Field," 32nd European Regional ITS Conference, Madrid 2023: Realising the digital decade in the European Union – Easier said than done? 277958, International Telecommunications Society (ITS).
    5. Martin, Simon & Rasch, Alexander, 2022. "Collusion by algorithm: The role of unobserved actions," DICE Discussion Papers 382, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    6. Werner, Tobias, 2023. "Algorithmic and Human Collusion," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277573, Verein für Socialpolitik / German Economic Association.
    7. Fourberg, Niklas & Marques-Magalhaes, Katrin & Wiewiorra, Lukas, 2022. "They are among us: Pricing behavior of algorithms in the field," WIK Working Papers 6, WIK Wissenschaftliches Institut für Infrastruktur und Kommunikationsdienste GmbH, Bad Honnef.
    8. Joseph E. Harrington, 2022. "The Effect of Outsourcing Pricing Algorithms on Market Competition," Management Science, INFORMS, vol. 68(9), pages 6889-6906, September.
    9. Justin P. Johnson & Andrew Rhodes & Matthijs Wildenbeest, 2023. "Platform Design When Sellers Use Pricing Algorithms," Econometrica, Econometric Society, vol. 91(5), pages 1841-1879, September.
    10. Hans-Theo Normann & Martin Sternberg, 2021. "Human-Algorithm Interaction: Algorithmic Pricing in Hybrid Laboratory Markets," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2021_11, Max Planck Institute for Research on Collective Goods, revised 13 Apr 2022.
    11. Martino Banchio & Giacomo Mantegazza, 2022. "Artificial Intelligence and Spontaneous Collusion," Papers 2202.05946, arXiv.org, revised Sep 2023.
    12. Gonzalo Ballestero, 2022. "Collusion and Artificial Intelligence: A Computational Experiment with Sequential Pricing Algorithms under Stochastic Costs," Working Papers 118, Red Nacional de Investigadores en Economía (RedNIE).
    13. Martin, Simon & Rasch, Alexander, 2024. "Demand forecasting, signal precision, and collusion with hidden actions," International Journal of Industrial Organization, Elsevier, vol. 92(C).
    14. Connor Douglas & Foster Provost & Arun Sundararajan, 2024. "Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete?," Papers 2411.16574, arXiv.org.
    15. Gonzalo Ballestero, 2021. "Collusion and Artificial Intelligence: A computational experiment with sequential pricing algorithms under stochastic costs," Young Researchers Working Papers 1, Universidad de San Andres, Departamento de Economia, revised Oct 2022.
    16. Calvano, Emilio & Calzolari, Giacomo & Denicolò, Vincenzo & Pastorello, Sergio, 2023. "Algorithmic collusion: Genuine or spurious?," International Journal of Industrial Organization, Elsevier, vol. 90(C).
    17. Zhang Xu & Wei Zhao, 2024. "On Mechanism Underlying Algorithmic Collusion," Papers 2409.01147, arXiv.org.
    18. Epivent, Andréa & Lambin, Xavier, 2024. "On algorithmic collusion and reward–punishment schemes," Economics Letters, Elsevier, vol. 237(C).
    19. Bernhard Kasberger & Simon Martin & Hans-Theo Normann & Tobias Werner, 2024. "Algorithmic Cooperation," CESifo Working Paper Series 11124, CESifo.
    20. John Asker & Chaim Fershtman & Ariel Pakes, 2024. "The impact of artificial intelligence design on pricing," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 276-304, March.

    More about this item

    JEL classification:

    • L1 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance
    • L4 - Industrial Organization - - Antitrust Issues and Policies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aep:anales:4587. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Juan Manuel Quintero (email available below). General contact details of provider: https://edirc.repec.org/data/aaeppea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.