IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2302.00586.html
   My bibliography  Save this paper

PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets

Author

Listed:
  • Shuo Sun
  • Molei Qin
  • Xinrun Wang
  • Bo An

Abstract

The financial markets, which involve more than $90 trillion market capitals, attract the attention of innumerable investors around the world. Recently, reinforcement learning in financial markets (FinRL) has emerged as a promising direction to train agents for making profitable investment decisions. However, the evaluation of most FinRL methods only focuses on profit-related measures and ignores many critical axes, which are far from satisfactory for financial practitioners to deploy these methods into real-world financial markets. Therefore, we introduce PRUDEX-Compass, which has 6 axes, i.e., Profitability, Risk-control, Universality, Diversity, rEliability, and eXplainability, with a total of 17 measures for a systematic evaluation. Specifically, i) we propose AlphaMix+ as a strong FinRL baseline, which leverages mixture-of-experts (MoE) and risk-sensitive approaches to make diversified risk-aware investment decisions, ii) we evaluate 8 FinRL methods in 4 long-term real-world datasets of influential financial markets to demonstrate the usage of our PRUDEX-Compass, iii) PRUDEX-Compass together with 4 real-world datasets, standard implementation of 8 FinRL methods and a portfolio management environment is released as public resources to facilitate the design and comparison of new FinRL methods. We hope that PRUDEX-Compass can not only shed light on future FinRL research to prevent untrustworthy results from stagnating FinRL into successful industry deployment but also provide a new challenging algorithm evaluation scenario for the reinforcement learning (RL) community.

Suggested Citation

  • Shuo Sun & Molei Qin & Xinrun Wang & Bo An, 2023. "PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets," Papers 2302.00586, arXiv.org, revised Mar 2023.
    • Handle: RePEc:arx:papers:2302.00586
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2302.00586
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Marimoutou, Velayoudoum & Raggad, Bechir & Trabelsi, Abdelwahed, 2009. "Extreme Value Theory and Value at Risk: Application to oil market," Energy Economics, Elsevier, vol. 31(4), pages 519-530, July.
    2. István Ábel & Pierre L. Siklos & István P. Székely, 1998. "Money and Finance in the Transition to a Market Economy," Books, Edward Elgar Publishing, number 830.
    3. Zhengyao Jiang & Dixing Xu & Jinjun Liang, 2017. "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," Papers 1706.10059, arXiv.org, revised Jul 2017.
    4. repec:bla:jfinan:v:58:y:2003:i:3:p:975-1008 is not listed on IDEAS
    5. Madhavan, Ananth, 2000. "Market microstructure: A survey," Journal of Financial Markets, Elsevier, vol. 3(3), pages 205-258, August.
    6. De Santis, Giorgio & imrohoroglu, Selahattin, 1997. "Stock returns and volatility in emerging financial markets," Journal of International Money and Finance, Elsevier, vol. 16(4), pages 561-579, August.
    7. Emenyonu, Emmanuel N. & Gray, Sidney J., 1996. "International accounting harmonization and the major developed stock market countries: An empirical study," The International Journal of Accounting, Elsevier, vol. 31(3), pages 269-279.
    8. Thomas Spooner & John Fearnley & Rahul Savani & Andreas Koukorinis, 2018. "Market Making via Reinforcement Learning," Papers 1804.04216, arXiv.org.
    9. Thomas Spooner & Rahul Savani, 2020. "Robust Market Making via Adversarial Reinforcement Learning," Papers 2003.01820, arXiv.org, revised Jul 2020.
    10. Campbell, John Y. & Lo, Andrew W. & MacKinlay, A. Craig & Whitelaw, Robert F., 1998. "The Econometrics Of Financial Markets," Macroeconomic Dynamics, Cambridge University Press, vol. 2(4), pages 559-562, December.
    11. Lars Brink & Bruce McCarl, 1978. "The Tradeoff between Expected Return and Risk Among Cornbelt Farmers," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 60(2), pages 259-263.
    12. Amit Goyal & Pedro Santa‐Clara, 2003. "Idiosyncratic Risk Matters!," Journal of Finance, American Finance Association, vol. 58(3), pages 975-1007, June.
    13. Yuchen Fang & Kan Ren & Weiqing Liu & Dong Zhou & Weinan Zhang & Jiang Bian & Yong Yu & Tie-Yan Liu, 2021. "Universal Trading for Order Execution with Oracle Policy Distillation," Papers 2103.10860, arXiv.org.
    14. Yunan Ye & Hengzhi Pei & Boxin Wang & Pin-Yu Chen & Yada Zhu & Jun Xiao & Bo Li, 2020. "Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States," Papers 2002.05780, arXiv.org.
    15. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    16. Ulrich Kirchner & Caroline Zunckel, 2011. "Measuring Portfolio Diversification," Papers 1102.4722, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wentao Zhang & Lingxuan Zhao & Haochong Xia & Shuo Sun & Jiaze Sun & Molei Qin & Xinyi Li & Yuqing Zhao & Yilei Zhao & Xinyu Cai & Longtao Zheng & Xinrun Wang & Bo An, 2024. "A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist," Papers 2402.18485, arXiv.org, revised Jun 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    2. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    3. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    4. Hui Niu & Siyuan Li & Jian Li, 2022. "MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization," Papers 2210.01774, arXiv.org.
    5. Khan, Khalid & Derindere Köseoğlu, Sinem, 2020. "Is palladium price in bubble?," Resources Policy, Elsevier, vol. 68(C).
    6. Alexander Jakob Dautel & Wolfgang Karl Härdle & Stefan Lessmann & Hsin-Vonn Seow, 2020. "Forex exchange rate forecasting using deep recurrent neural networks," Digital Finance, Springer, vol. 2(1), pages 69-96, September.
    7. Stephen Kawas & Everton Dockery, 2023. "What do we know about the stock markets’ reaction to regulatory announcements regarding financial institutions? Evidence from UK financial institutions," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 31-67, January.
    8. Xu, Yingying & Lien, Donald, 2022. "Which affects stock performances more, words or deeds of the key person?," International Review of Financial Analysis, Elsevier, vol. 84(C).
    9. Stefan Avdjiev & Maximilian Jager, 2022. "Bank opacity - patterns and implications," BIS Working Papers 992, Bank for International Settlements.
    10. Riehl, Kevin & Kiesel, Florian & Schiereck, Dirk, 2022. "Political and Socioeconomic Factors That Determine the Financial Outcome of Successful Green Innovation," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 132099, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    11. Naderi Semiromi, Hamed & Lessmann, Stefan & Peters, Wiebke, 2020. "News will tell: Forecasting foreign exchange rates based on news story events in the economy calendar," The North American Journal of Economics and Finance, Elsevier, vol. 52(C).
    12. Doan, Minh Phuong & Sercu, Piet, 2021. "Modelling multiperiod patterns in stock-market reactions to events, with an application to serial acquisitions," International Review of Financial Analysis, Elsevier, vol. 77(C).
    13. Iwao Maeda & David deGraw & Michiharu Kitano & Hiroyasu Matsushima & Hiroki Sakaji & Kiyoshi Izumi & Atsuo Kato, 2020. "Deep Reinforcement Learning in Agent Based Financial Market Simulation," JRFM, MDPI, vol. 13(4), pages 1-17, April.
    14. Paolo Maranzano & Matteo Maria Pelagatti, 2022. "Spatio-temporal Event Studies for Air Quality Assessment under Cross-sectional Dependence," Papers 2210.17529, arXiv.org.
    15. Laurel PASRICHA & Neelam DHANDA, 2022. "The past, the present and the prospective future of efficient market hypothesis: a theoretical and empirical investigation of international stock markets," Theoretical and Applied Economics, Asociatia Generala a Economistilor din Romania / Editura Economica, vol. 0(2(631), S), pages 89-106, Summer.
    16. Shuo Sun & Wanqi Xue & Rundong Wang & Xu He & Junlei Zhu & Jian Li & Bo An, 2021. "DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities," Papers 2201.09058, arXiv.org, revised Aug 2022.
    17. Nelson Vadori & Leo Ardon & Sumitra Ganesh & Thomas Spooner & Selim Amrouni & Jared Vann & Mengda Xu & Zeyu Zheng & Tucker Balch & Manuela Veloso, 2022. "Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations," Papers 2210.07184, arXiv.org, revised Aug 2023.
    18. Wahab, Bashir A. & Adewuyi, Adeolu O., 2021. "Analysis of major properties of metal prices using new methods: Structural breaks, non-linearity, stationarity and bubbles," Resources Policy, Elsevier, vol. 74(C).
    19. Zhenhan Huang & Fumihide Tanaka, 2021. "MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management," Papers 2102.03502, arXiv.org, revised Feb 2022.
    20. Shu, Lei & Lu, Feiyang & Chen, Yu, 2023. "Robust forecasting with scaled independent component analysis," Finance Research Letters, Elsevier, vol. 51(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2302.00586. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.