IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2406.14537.html
   My bibliography  Save this paper

MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading

Author

Listed:
  • Chuqiao Zong
  • Chaojie Wang
  • Molei Qin
  • Lei Feng
  • Xinrun Wang
  • Bo An

Abstract

High-frequency trading (HFT) that executes algorithmic trading in short time scales, has recently occupied the majority of cryptocurrency market. Besides traditional quantitative trading methods, reinforcement learning (RL) has become another appealing approach for HFT due to its terrific ability of handling high-dimensional financial data and solving sophisticated sequential decision-making problems, \emph{e.g.,} hierarchical reinforcement learning (HRL) has shown its promising performance on second-level HFT by training a router to select only one sub-agent from the agent pool to execute the current transaction. However, existing RL methods for HFT still have some defects: 1) standard RL-based trading agents suffer from the overfitting issue, preventing them from making effective policy adjustments based on financial context; 2) due to the rapid changes in market conditions, investment decisions made by an individual agent are usually one-sided and highly biased, which might lead to significant loss in extreme markets. To tackle these problems, we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, \emph{a.k.a.} MacroHFT, which consists of two training phases: 1) we first train multiple types of sub-agents with the market data decomposed according to various financial indicators, specifically market trend and volatility, where each agent owns a conditional adapter to adjust its trading policy according to market conditions; 2) then we train a hyper-agent to mix the decisions from these sub-agents and output a consistently profitable meta-policy to handle rapid market fluctuations, equipped with a memory mechanism to enhance the capability of decision-making. Extensive experiments on various cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.

Suggested Citation

  • Chuqiao Zong & Chaojie Wang & Molei Qin & Lei Feng & Xinrun Wang & Bo An, 2024. "MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading," Papers 2406.14537, arXiv.org.
  • Handle: RePEc:arx:papers:2406.14537
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2406.14537
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fan Fang & Carmine Ventre & Michail Basios & Leslie Kanthan & David Martinez-Rego & Fan Wu & Lingbo Li, 2022. "Cryptocurrency trading: a comprehensive survey," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 8(1), pages 1-59, December.
    2. Chuheng Zhang & Yitong Duan & Xiaoyu Chen & Jianyu Chen & Jian Li & Li Zhao, 2023. "Towards Generalizable Reinforcement Learning for Trade Execution," Papers 2307.11685, arXiv.org.
    3. Tian Zhu & Wei Zhu, 2022. "Quantitative Trading through Random Perturbation Q-Network with Nonlinear Transaction Costs," Stats, MDPI, vol. 5(2), pages 1-15, June.
    4. Chordia, Tarun & Roll, Richard & Subrahmanyam, Avanidhar, 2002. "Order imbalance, liquidity, and market returns," Journal of Financial Economics, Elsevier, vol. 65(1), pages 111-130, July.
    5. Zura Kakushadze, 2016. "101 Formulaic Alphas," Papers 1601.00991, arXiv.org, revised Mar 2016.
    6. Madhavan, Ananth, 2000. "Market microstructure: A survey," Journal of Financial Markets, Elsevier, vol. 3(3), pages 205-258, August.
    7. Molei Qin & Shuo Sun & Wentao Zhang & Haochong Xia & Xinrun Wang & Bo An, 2023. "EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading," Papers 2309.12891, arXiv.org.
    8. Fan Fang & Carmine Ventre & Michail Basios & Leslie Kanthan & Lingbo Li & David Martinez-Regoband & Fan Wu, 2020. "Cryptocurrency Trading: A Comprehensive Survey," Papers 2003.11352, arXiv.org, revised Jan 2022.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. I-Chun Tsai, 2024. "Features of different asset types and extreme risk transmission during the COVID-19 crisis," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 10(1), pages 1-42, December.
    2. Molei Qin & Shuo Sun & Wentao Zhang & Haochong Xia & Xinrun Wang & Bo An, 2023. "EarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading," Papers 2309.12891, arXiv.org.
    3. Soria, Jorge & Moya, Jorge & Mohazab, Amin, 2023. "Optimal mining in proof-of-work blockchain protocols," Finance Research Letters, Elsevier, vol. 53(C).
    4. Siu Hin Tang & Mathieu Rosenbaum & Chao Zhou, 2023. "Forecasting Volatility with Machine Learning and Rough Volatility: Example from the Crypto-Winter," Papers 2311.04727, arXiv.org, revised Feb 2024.
    5. Yichen Luo & Yebo Feng & Jiahua Xu & Paolo Tasca & Yang Liu, 2025. "LLM-Powered Multi-Agent System for Automated Crypto Portfolio Management," Papers 2501.00826, arXiv.org, revised Jan 2025.
    6. Gang Kou & Yang Lu, 2025. "FinTech: a literature review of emerging financial technologies and applications," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 11(1), pages 1-34, December.
    7. Husam Rjoub & Tomiwa Sunday Adebayo & Dervis Kirikkaleli, 2023. "Blockchain technology-based FinTech banking sector involvement using adaptive neuro-fuzzy-based K-nearest neighbors algorithm," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-23, December.
    8. Yiuman Tse & Brian C. McTier & John K. Wald, 2011. "Do Stock Markets Catch the Flu? We examine the impact of influenza on the U.S. stock market. A higher incidence of flu is associated with decreased trading, decreased volatility, and higher bid-ask sp," Working Papers 0004, College of Business, University of Texas at San Antonio.
    9. Esther Calderon-Monge & Domingo Ribeiro-Soriano, 2024. "The role of digitalization in business and management: a systematic literature review," Review of Managerial Science, Springer, vol. 18(2), pages 449-491, February.
    10. Mahmut Bağcı & Pınar Kaya Soylu, 2024. "Optimal portfolio selection with volatility information for a high frequency rebalancing algorithm," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 10(1), pages 1-28, December.
    11. Marcin Wątorek & Jarosław Kwapień & Stanisław Drożdż, 2022. "Multifractal Cross-Correlations of Bitcoin and Ether Trading Characteristics in the Post-COVID-19 Time," Future Internet, MDPI, vol. 14(7), pages 1-15, July.
    12. Laurens Swinkels, 2023. "Empirical evidence on the ownership and liquidity of real estate tokens," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-29, December.
    13. Luyao Zhang & Tianyu Wu & Saad Lahrichi & Carlos-Gustavo Salas-Flores & Jiayi Li, 2022. "A Data Science Pipeline for Algorithmic Trading: A Comparative Study of Applications for Finance and Cryptoeconomics," Papers 2206.14932, arXiv.org.
    14. Kirimhan, Destan, 2023. "Importance of anti-money laundering regulations among prosumers for a cybersecure decentralized finance," Journal of Business Research, Elsevier, vol. 157(C).
    15. Kate Murray & Andrea Rossi & Diego Carraro & Andrea Visentin, 2023. "On Forecasting Cryptocurrency Prices: A Comparison of Machine Learning, Deep Learning, and Ensembles," Forecasting, MDPI, vol. 5(1), pages 1-14, January.
    16. Jules Clement Mba, 2024. "Assessing portfolio vulnerability to systemic risk: a vine copula and APARCH-DCC approach," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 10(1), pages 1-36, December.
    17. Luis Lorenzo & Javier Arroyo, 2023. "Online risk-based portfolio allocation on subsets of crypto assets applying a prototype-based clustering algorithm," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-40, December.
    18. Wujun Lv & Tao Pang & Xiaobao Xia & Jingzhou Yan, 2023. "Dynamic portfolio choice with uncertain rare-events risk in stock and cryptocurrency markets," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-28, December.
    19. Wolfgang Breuer & Andreas Knetsch, 2023. "Recent trends in the digitalization of finance and accounting," Journal of Business Economics, Springer, vol. 93(9), pages 1451-1461, November.
    20. Wei Xu & Daning Hu & Karl Reiner Lang & J. Leon Zhao, 2022. "Blockchain and digital finance," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 8(1), pages 1-4, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2406.14537. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.