IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2112.06753.html
   My bibliography  Save this paper

FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance

Author

Listed:
  • Xiao-Yang Liu
  • Jingyang Rui
  • Jiechao Gao
  • Liuqing Yang
  • Hongyang Yang
  • Zhaoran Wang
  • Christina Dan Wang
  • Jian Guo

Abstract

Deep reinforcement learning (DRL) has shown huge potentials in building financial market simulators recently. However, due to the highly complex and dynamic nature of real-world markets, raw historical financial data often involve large noise and may not reflect the future of markets, degrading the fidelity of DRL-based market simulators. Moreover, the accuracy of DRL-based market simulators heavily relies on numerous and diverse DRL agents, which increases demand for a universe of market environments and imposes a challenge on simulation speed. In this paper, we present a FinRL-Meta framework that builds a universe of market environments for data-driven financial reinforcement learning. First, FinRL-Meta separates financial data processing from the design pipeline of DRL-based strategy and provides open-source data engineering tools for financial big data. Second, FinRL-Meta provides hundreds of market environments for various trading tasks. Third, FinRL-Meta enables multiprocessing simulation and training by exploiting thousands of GPU cores. Our codes are available online at https://github.com/AI4Finance-Foundation/FinRL-Meta.

Suggested Citation

  • Xiao-Yang Liu & Jingyang Rui & Jiechao Gao & Liuqing Yang & Hongyang Yang & Zhaoran Wang & Christina Dan Wang & Jian Guo, 2021. "FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance," Papers 2112.06753, arXiv.org, revised Mar 2022.
    • Handle: RePEc:arx:papers:2112.06753
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2112.06753
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Michael Karpe & Jin Fang & Zhongyao Ma & Chen Wang, 2020. "Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation," Papers 2006.05574, arXiv.org, revised Sep 2020.
    2. Johann Lussange & Ivan Lazarevich & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2021. "Modelling Stock Markets by Multi-agent Reinforcement Learning," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 113-147, January.
    3. Raberto, Marco & Cincotti, Silvano & Focardi, Sergio M. & Marchesi, Michele, 2001. "Agent-based simulation of a financial market," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 299(1), pages 319-327.
    4. Agrim Gupta & Silvio Savarese & Surya Ganguli & Li Fei-Fei, 2021. "Embodied intelligence via learning and evolution," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    5. Suman Ravuri & Karel Lenc & Matthew Willson & Dmitry Kangin & Remi Lam & Piotr Mirowski & Megan Fitzsimons & Maria Athanassiadou & Sheleem Kashem & Sam Madge & Rachel Prudden & Amol Mandhane & Aidan C, 2021. "Skilful precipitation nowcasting using deep generative models of radar," Nature, Nature, vol. 597(7878), pages 672-677, September.
    6. Xiao-Yang Liu & Hongyang Yang & Jiechao Gao & Christina Dan Wang, 2021. "FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance," Papers 2111.09395, arXiv.org.
    7. Wenhang Bao & Xiao-yang Liu, 2019. "Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis," Papers 1906.11046, arXiv.org.
    8. Tidor-Vlad Pricope, 2021. "Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review," Papers 2106.00123, arXiv.org.
    9. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    10. Maarten P. Scholl & Anisoara Calinescu & J. Doyne Farmer, 2021. "How market ecology explains market malfunction," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(26), pages 2015574118-, June.
    11. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiao-Yang Liu & Ziyi Xia & Jingyang Rui & Jiechao Gao & Hongyang Yang & Ming Zhu & Christina Dan Wang & Zhaoran Wang & Jian Guo, 2022. "FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning," Papers 2211.03107, arXiv.org.
    2. Jinan Zou & Qingying Zhao & Yang Jiao & Haiyao Cao & Yanxi Liu & Qingsen Yan & Ehsan Abbasnejad & Lingqiao Liu & Javen Qinfeng Shi, 2022. "Stock Market Prediction via Deep Learning Techniques: A Survey," Papers 2212.12717, arXiv.org, revised Feb 2023.
    3. Mao Guan & Xiao-Yang Liu, 2021. "Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach," Papers 2111.03995, arXiv.org, revised Dec 2021.
    4. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    5. Xiao-Yang Liu & Hongyang Yang & Jiechao Gao & Christina Dan Wang, 2021. "FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance," Papers 2111.09395, arXiv.org.
    6. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    7. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    8. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    9. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay & Jamal Atif, 2020. "AAMDRL: Augmented Asset Management with Deep Reinforcement Learning," Papers 2010.08497, arXiv.org.
    10. E. Samanidou & E. Zschischang & D. Stauffer & T. Lux, 2001. "Microscopic Models of Financial Markets," Papers cond-mat/0110354, arXiv.org.
    11. Sina Montazeri & Akram Mirzaeinia & Haseebullah Jumakhan & Amir Mirzaeinia, 2024. "CNN-DRL for Scalable Actions in Finance," Papers 2401.06179, arXiv.org.
    12. Thiago W. Alves & Ionuţ Florescu & Dragoş Bozdog, 2023. "Insights on the Statistics and Market Behavior of Frequent Batch Auctions," Mathematics, MDPI, vol. 11(5), pages 1-26, March.
    13. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep equal risk pricing of financial derivatives with non-translation invariant risk measures," Papers 2107.11340, arXiv.org.
    14. Marco Raberto & Silvano Cincotti & Sergio Focardi & Michele Marchesi, 2003. "Traders' Long-Run Wealth in an Artificial Financial Market," Computational Economics, Springer;Society for Computational Economics, vol. 22(2), pages 255-272, October.
    15. Silvano Cincotti & Marco Raberto & Andrea Teglio, 2022. "Why do we need agent-based macroeconomics?," Review of Evolutionary Political Economy, Springer, vol. 3(1), pages 5-29, April.
    16. Linda Ponta & Silvano Cincotti, 2018. "Traders’ Networks of Interactions and Structural Properties of Financial Markets: An Agent-Based Approach," Complexity, Hindawi, vol. 2018, pages 1-9, January.
    17. Olschewski, Sebastian & Diao, Linan & Rieskamp, Jörg, 2021. "Reinforcement learning about asset variability and correlation in repeated portfolio decisions," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    18. Silvano Cincotti & Wolfram Elsner & Nathalie Lazaric & Anastasia Nesvetailova & Engelbert Stockhammer, 2020. "Towards an evolutionary political economy. Editorial to the inaugural issue of the Review of Evolutionary Political Economy REPE," Review of Evolutionary Political Economy, Springer, vol. 1(1), pages 1-12, May.
    19. Tubbenhauer, Tobias & Fieberg, Christian & Poddig, Thorsten, 2021. "Multi-agent-based VaR forecasting," Journal of Economic Dynamics and Control, Elsevier, vol. 131(C).
    20. Derveeuw, Julien & Beaufils, Bruno & Mathieu, Philippe & Brandouy, Olivier, 2007. "Testing double auction as a component within a generic market model architecture," MPRA Paper 4918, University Library of Munich, Germany.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2112.06753. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.