IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-04790290.html
   My bibliography  Save this paper

Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study

Author

Listed:
  • Johann Lussange

    (Group for Neural Theory [Paris] - LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale - IEC - Labex Institut d'étude de la cognition - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres, LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale)

  • Stefano Vrizzi

    (Group for Neural Theory [Paris] - LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale - IEC - Labex Institut d'étude de la cognition - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres, LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale)

  • Stefano Palminteri

    (LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale, Center for Cognition and Decision Making - HSE - Vysšaja škola èkonomiki = National Research University Higher School of Economics [Moscow])

  • Boris Gutkin

    (Group for Neural Theory [Paris] - LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale - IEC - Labex Institut d'étude de la cognition - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres, LNC2 - Laboratoire de Neurosciences Cognitives & Computationnelles - DEC - Département d'Etudes Cognitives - ENS-PSL - ENS-PSL - École normale supérieure - Paris - PSL - Université Paris Sciences et Lettres - INSERM - Institut National de la Santé et de la Recherche Médicale, Center for Cognition and Decision Making - HSE - Vysšaja škola èkonomiki = National Research University Higher School of Economics [Moscow])

Abstract

Recent advances in the field of machine learning have yielded novel research perspectives in behavioural economics and financial markets microstructure studies. In this paper we study the impact of individual trader leaning characteristics on markets using a stock market simulator designed with a multi-agent architecture. Each agent, representing an autonomous investor, trades stocks through reinforcement learning, using a centralized doubleauction limit order book. This approach allows us to study the impact of individual trader traits on the whole stock market at the mesoscale in a bottom-up approach. We chose to test three trader trait aspects: agent learning rate increases, herding behaviour and random trading. As hypothesized, we find that larger learning rates significantly increase the number of crashes. We also find that herding behaviour undermines market stability, while random trading tends to preserve it.

Suggested Citation

  • Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study," Post-Print hal-04790290, HAL.
    • Handle: RePEc:hal:journl:hal-04790290
    DOI: 10.1371/journal.pone.0301141
    Note: View the original document on HAL open archive server: https://hal.science/hal-04790290v1
    as

    Download full text from publisher

    File URL: https://hal.science/hal-04790290v1/document
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0301141?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Roberto Mota Navarro & Hern'an Larralde Ridaura, 2016. "A detailed heterogeneous agent model for a single asset financial market with trading via an order book," Papers 1601.00229, arXiv.org, revised Jul 2016.
    2. John Y. Campbell & Sanford J. Grossman & Jiang Wang, 1993. "Trading Volume and Serial Correlation in Stock Returns," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(4), pages 905-939.
    3. Stephen J. Hardiman & Nicolas Bercot & Jean-Philippe Bouchaud, 2013. "Critical reflexivity in financial markets: a Hawkes process analysis," Papers 1302.1405, arXiv.org, revised Jun 2013.
    4. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    5. Salle, Isabelle & Yildizoglu, Murat & Zumpe, Martin & Sénégas, Marc-Alexandre, 2017. "Coordination through social learning in a general equilibrium model," Journal of Economic Behavior & Organization, Elsevier, vol. 141(C), pages 64-82.
    6. Stefano Palminteri & Mehdi Khamassi & Mateus Joffily & Giorgio Coricelli, 2015. "Contextual modulation of value signals in reward and punishment learning," Nature Communications, Nature, vol. 6(1), pages 1-14, November.
    7. Grossman, Sanford J & Stiglitz, Joseph E, 1980. "On the Impossibility of Informationally Efficient Markets," American Economic Review, American Economic Association, vol. 70(3), pages 393-408, June.
    8. Sylvain Barde, 2015. "Direct calibration and comparison of agent-based herding models of financial markets," Studies in Economics 1507, School of Economics, University of Kent.
    9. Salle, Isabelle & Yildizoglu, Murat & Zumpe, Martin & Sénégas, Marc-Alexandre, 2017. "Coordination through social learning in a general equilibrium model," Journal of Economic Behavior & Organization, Elsevier, vol. 141(C), pages 64-82.
    10. Cont, Rama & Bouchaud, Jean-Philipe, 2000. "Herd Behavior And Aggregate Fluctuations In Financial Markets," Macroeconomic Dynamics, Cambridge University Press, vol. 4(2), pages 170-196, June.
    11. Justin Sirignano & Rama Cont, 2019. "Universal features of price formation in financial markets: perspectives from deep learning," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1449-1459, September.
    12. Charles Cao & Yong Chen & William N. Goetzmann & Bing Liang, 2018. "Hedge Funds and Stock Price Formation," Financial Analysts Journal, Taylor & Francis Journals, vol. 74(3), pages 54-68, July.
    13. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    14. Jan A. Lipski & Ryszard Kutner, 2013. "Agent-Based Stock Market Model with Endogenous Agents' Impact," Papers 1310.0762, arXiv.org, revised Dec 2013.
    15. Marcus Cordi & Damien Challet & Ioane Muni Toke, 2017. "Testing the causality of Hawkes processes with time reversal," Papers 1709.08516, arXiv.org.
    16. Westerhoff Frank H., 2008. "The Use of Agent-Based Financial Market Models to Test the Effectiveness of Regulatory Policies," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 228(2-3), pages 195-227, April.
    17. Tomas Ramanauskas & Aleksandras Vytautas Rutkauskas, 2009. "Building an Artificial Stock Market Populated by Reinforcement-Learning Agents," Bank of Lithuania Working Paper Series 6, Bank of Lithuania.
    18. Weibing Huang & Charles-Albert Lehalle & Mathieu Rosenbaum, 2015. "Simulating and Analyzing Order Book Data: The Queue-Reactive Model," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 107-122, March.
    19. Gualdi, Stanislao & Tarzia, Marco & Zamponi, Francesco & Bouchaud, Jean-Philippe, 2015. "Tipping points in macroeconomic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 50(C), pages 29-61.
    20. Gode, Dhananjay K & Sunder, Shyam, 1993. "Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality," Journal of Political Economy, University of Chicago Press, vol. 101(1), pages 119-137, February.
    21. Orlean, Andre, 1995. "Bayesian interactions and collective dynamics of opinion: Herd behavior and mimetic contagion," Journal of Economic Behavior & Organization, Elsevier, vol. 28(2), pages 257-274, October.
    22. Pramod Kumar Naik & Rangan Gupta & Puja Padhi, 2018. "The Relationship Between Stock Market Volatility And Trading Volume: Evidence From South Africa," Journal of Developing Areas, Tennessee State University, College of Business, vol. 52(1), pages 99-114, January-M.
    23. Stephen Hardiman & Nicolas Bercot & Jean-Philippe Bouchaud, 2013. "Critical reflexivity in financial markets: a Hawkes process analysis," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 86(10), pages 1-9, October.
    24. Anna Dodonova & Yuri Khoroshilov, 2018. "Private information in futures markets: An experimental study," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 39(1), pages 65-70, January.
    25. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    26. Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    27. Stefano Palminteri & Germain Lefebvre & Emma J Kilford & Sarah-Jayne Blakemore, 2017. "Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing," PLOS Computational Biology, Public Library of Science, vol. 13(8), pages 1-22, August.
    28. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    29. Jonathan Lewellen, 2002. "Momentum and Autocorrelation in Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 15(2), pages 533-564, March.
    30. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    31. Germain Lefebvre & Maël Lebreton & Florent Meyniel & Sacha Bourgeois-Gironde & Stefano Palminteri, 2017. "Behavioural and neural characterization of optimistic reinforcement learning," Nature Human Behaviour, Nature, vol. 1(4), pages 1-9, April.
    32. I. Momennejad & E. M. Russek & J. H. Cheong & M. M. Botvinick & N. D. Daw & S. J. Gershman, 2017. "The successor representation in human reinforcement learning," Nature Human Behaviour, Nature, vol. 1(9), pages 680-692, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johann Lussange & Stefano Vrizzi & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2023. "Stock Price Formation: Precepts from a Multi-Agent Reinforcement Learning Model," Computational Economics, Springer;Society for Computational Economics, vol. 61(4), pages 1523-1544, April.
    2. Johann Lussange & Ivan Lazarevich & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2021. "Modelling Stock Markets by Multi-agent Reinforcement Learning," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 113-147, January.
    3. Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Modelling crypto markets by multi-agent reinforcement learning," Papers 2402.10803, arXiv.org.
    4. Johann Lussange & Boris Gutkin, 2023. "Order book regulatory impact on stock market quality: a multi-agent reinforcement learning perspective," Papers 2302.04184, arXiv.org.
    5. Jovanovic, Franck & Schinckus, Christophe, 2017. "Econophysics and Financial Economics: An Emerging Dialogue," OUP Catalogue, Oxford University Press, number 9780190205034.
    6. Torsten Trimborn & Philipp Otte & Simon Cramer & Maximilian Beikirch & Emma Pabich & Martin Frank, 2020. "SABCEMM: A Simulator for Agent-Based Computational Economic Market Models," Computational Economics, Springer;Society for Computational Economics, vol. 55(2), pages 707-744, February.
    7. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    8. Miko{l}aj Bi'nkowski & Charles-Albert Lehalle, 2018. "Endogeneous Dynamics of Intraday Liquidity," Papers 1811.03766, arXiv.org.
    9. Arvid Oskar Ivar Hoffmann & Wander Jager & J. H. Von Eije, 2007. "Social Simulation of Stock Markets: Taking It to the Next Level," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 10(2), pages 1-7.
    10. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    11. Luis Goncalves de Faria, 2022. "An Agent-Based Model With Realistic Financial Time Series: A Method for Agent-Based Models Validation," Papers 2206.09772, arXiv.org.
    12. Didier SORNETTE, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based Models," Swiss Finance Institute Research Paper Series 14-25, Swiss Finance Institute.
    13. Ashok Chanabasangouda Patil & Shailesh Rastogi, 2019. "Time-Varying Price–Volume Relationship and Adaptive Market Efficiency: A Survey of the Empirical Literature," JRFM, MDPI, vol. 12(2), pages 1-18, June.
    14. Noemi Schmitt & Ivonne Schwartz & Frank Westerhoff, 2022. "Heterogeneous speculators and stock market dynamics: a simple agent-based computational model," The European Journal of Finance, Taylor & Francis Journals, vol. 28(13-15), pages 1263-1282, October.
    15. D. Sornette, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based models," Papers 1404.0243, arXiv.org.
    16. Gao-Feng Gu & Xiong Xiong & Hai-Chuan Xu & Wei Zhang & Yongjie Zhang & Wei Chen & Wei-Xing Zhou, 2021. "An empirical behavioral order-driven model with price limit rules," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 7(1), pages 1-24, December.
    17. Vivien Lespagnol & Juliette Rouchier, 2014. "Trading Volume and Market Efficiency: An Agent Based Model with Heterogenous Knowledge about Fundamentals," Working Papers halshs-00997573, HAL.
    18. Tedeschi, Gabriele & Iori, Giulia & Gallegati, Mauro, 2012. "Herding effects in order driven markets: The rise and fall of gurus," Journal of Economic Behavior & Organization, Elsevier, vol. 81(1), pages 82-96.
    19. Bohl, Martin T. & Pütz, Alexander & Sulewski, Christoph, 2021. "Speculation and the informational efficiency of commodity futures markets," Journal of Commodity Markets, Elsevier, vol. 23(C).
    20. Detlef Seese & Christof Weinhardt & Frank Schlottmann (ed.), 2008. "Handbook on Information Technology in Finance," International Handbooks on Information Systems, Springer, number 978-3-540-49487-4, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-04790290. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.