IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2502.09172.html
   My bibliography  Save this paper

LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data

Author

Listed:
  • Peer Nagy
  • Sascha Frey
  • Kang Li
  • Bidipta Sarkar
  • Svitlana Vyetrenko
  • Stefan Zohren
  • Ani Calinescu
  • Jakob Foerster

Abstract

While financial data presents one of the most challenging and interesting sequence modelling tasks due to high noise, heavy tails, and strategic interactions, progress in this area has been hindered by the lack of consensus on quantitative evaluation paradigms. To address this, we present LOB-Bench, a benchmark, implemented in python, designed to evaluate the quality and realism of generative message-by-order data for limit order books (LOB) in the LOBSTER format. Our framework measures distributional differences in conditional and unconditional statistics between generated and real LOB data, supporting flexible multivariate statistical evaluation. The benchmark also includes features commonly used LOB statistics such as spread, order book volumes, order imbalance, and message inter-arrival times, along with scores from a trained discriminator network. Lastly, LOB-Bench contains "market impact metrics", i.e. the cross-correlations and price response functions for specific events in the data. We benchmark generative autoregressive state-space models, a (C)GAN, as well as a parametric LOB model and find that the autoregressive GenAI approach beats traditional model classes.

Suggested Citation

  • Peer Nagy & Sascha Frey & Kang Li & Bidipta Sarkar & Svitlana Vyetrenko & Stefan Zohren & Ani Calinescu & Jakob Foerster, 2025. "LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data," Papers 2502.09172, arXiv.org.
    • Handle: RePEc:arx:papers:2502.09172
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2502.09172
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rama Cont & Arseniy Kukanov & Sasha Stoikov, 2014. "The Price Impact of Order Book Events," Journal of Financial Econometrics, Oxford University Press, vol. 12(1), pages 47-88.
    2. Rama Cont & Sasha Stoikov & Rishi Talreja, 2010. "A Stochastic Model for Order Book Dynamics," Operations Research, INFORMS, vol. 58(3), pages 549-563, June.
    3. Zoltán Eisler & Jean-Philippe Bouchaud & Julien Kockelkoren, 2012. "The price impact of order book events: market orders, limit orders and cancellations," Quantitative Finance, Taylor & Francis Journals, vol. 12(9), pages 1395-1419, September.
    4. Bence Toth & Zoltan Eisler & Jean-Philippe Bouchaud, 2016. "The square-root impact law also holds for option markets," Papers 1602.03043, arXiv.org.
    5. Peer Nagy & Sascha Frey & Silvia Sapora & Kang Li & Anisoara Calinescu & Stefan Zohren & Jakob Foerster, 2023. "Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network," Papers 2309.00638, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fabrizio Lillo, 2021. "Order flow and price formation," Papers 2105.00521, arXiv.org.
    2. Federico Gonzalez & Mark Schervish, 2017. "Instantaneous order impact and high-frequency strategy optimization in limit order books," Papers 1707.01167, arXiv.org, revised Oct 2017.
    3. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Papers 1803.06917, arXiv.org.
    4. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Working Papers hal-01754054, HAL.
    5. Mircea BAHNA & Cosmin-Octavian CEPOI & Bogdan Andrei DUMITRESCU & Virgil DAMIAN, 2018. "Estimating the Price Impact of Market Orders on the Bucharest Stock Exchange," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(4), pages 120-133, December.
    6. Leonardo Berti & Bardh Prenkaj & Paola Velardi, 2025. "TRADES: Generating Realistic Market Simulations with Diffusion Models," Papers 2502.07071, arXiv.org, revised Feb 2025.
    7. Filip Stanek & Jiri Kukacka, 2018. "The Impact of the Tobin Tax in a Heterogeneous Agent Model of the Foreign Exchange Market," Computational Economics, Springer;Society for Computational Economics, vol. 51(4), pages 865-892, April.
    8. Antonio Briola & Silvia Bartolucci & Tomaso Aste, 2024. "Deep Limit Order Book Forecasting," Papers 2403.09267, arXiv.org, revised Jun 2024.
    9. Zijian Shi & John Cartlidge, 2024. "Neural stochastic agent‐based limit order book simulation with neural point process and diffusion probabilistic model," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 31(2), June.
    10. Jonathan A. Chávez Casillas, 2024. "A Time-Dependent Markovian Model of a Limit Order Book," Computational Economics, Springer;Society for Computational Economics, vol. 63(2), pages 679-709, February.
    11. Philippe Bergault & Enzo Cogn'eville, 2024. "Simulating and analyzing a sparse order book: an application to intraday electricity markets," Papers 2410.06839, arXiv.org.
    12. Weibing Huang & Charles-Albert Lehalle & Mathieu Rosenbaum, 2015. "Simulating and Analyzing Order Book Data: The Queue-Reactive Model," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 107-122, March.
    13. Tucker Hybinette Balch & Mahmoud Mahfouz & Joshua Lockhart & Maria Hybinette & David Byrd, 2019. "How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?," Papers 1906.12010, arXiv.org.
    14. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    15. Xiaofei Lu & Frédéric Abergel, 2017. "Limit order book modelling with high dimensional Hawkes processes," Working Papers hal-01512430, HAL.
    16. Olivier Guéant, 2016. "The Financial Mathematics of Market Liquidity: From Optimal Execution to Market Making," Post-Print hal-01393136, HAL.
    17. Gao, Xuefeng & Xu, Tianrun, 2022. "Order scoring, bandit learning and order cancellations," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    18. Arzandeh, Mehdi & Frank, Julieta, 2017. "The Information Content of the Limit Order Book," 7th Annual Canadian Agri-Food Policy Conference, January 11-13, 2017, Ottawa, ON 253251, Canadian Agricultural Economics Society.
    19. Rama Cont & Marvin S. Mueller, 2019. "A stochastic partial differential equation model for limit order book dynamics," Papers 1904.03058, arXiv.org, revised May 2021.
    20. Korolev, V.Yu. & Chertok, A.V. & Korchagin, A.Yu. & Zeifman, A.I., 2015. "Modeling high-frequency order flow imbalance by functional limit theorems for two-sided risk processes," Applied Mathematics and Computation, Elsevier, vol. 253(C), pages 224-241.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2502.09172. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.