IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2105.03844.html
   My bibliography  Save this paper

Reinforcement Learning with Expert Trajectory For Quantitative Trading

Author

Listed:
  • Sihang Chen
  • Weiqi Luo
  • Chao Yu

Abstract

In recent years, quantitative investment methods combined with artificial intelligence have attracted more and more attention from investors and researchers. Existing related methods based on the supervised learning are not very suitable for learning problems with long-term goals and delayed rewards in real futures trading. In this paper, therefore, we model the price prediction problem as a Markov decision process (MDP), and optimize it by reinforcement learning with expert trajectory. In the proposed method, we employ more than 100 short-term alpha factors instead of price, volume and several technical factors in used existing methods to describe the states of MDP. Furthermore, unlike DQN (deep Q-learning) and BC (behavior cloning) in related methods, we introduce expert experience in training stage, and consider both the expert-environment interaction and the agent-environment interaction to design the temporal difference error so that the agents are more adaptable for inevitable noise in financial data. Experimental results evaluated on share price index futures in China, including IF (CSI 300) and IC (CSI 500), show that the advantages of the proposed method compared with three typical technical analysis and two deep leaning based methods.

Suggested Citation

  • Sihang Chen & Weiqi Luo & Chao Yu, 2021. "Reinforcement Learning with Expert Trajectory For Quantitative Trading," Papers 2105.03844, arXiv.org.
    • Handle: RePEc:arx:papers:2105.03844
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2105.03844
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Deep Reinforcement Learning for Trading," Papers 1911.10107, arXiv.org.
    2. Tony Guida, 2019. "Big Data and Machine Learning in Quantitative Investment," Post-Print hal-02298299, HAL.
    3. Fuli Feng & Huimin Chen & Xiangnan He & Ji Ding & Maosong Sun & Tat-Seng Chua, 2018. "Enhancing Stock Movement Prediction with Adversarial Training," Papers 1810.09936, arXiv.org, revised Jun 2019.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Uta Pigorsch & Sebastian Schafer, 2021. "High-Dimensional Stock Portfolio Trading with Deep Reinforcement Learning," Papers 2112.04755, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    2. Linyi Yang & Yingpeng Ma & Yue Zhang, 2023. "Measuring Consistency in Text-based Financial Forecasting Models," Papers 2305.08524, arXiv.org, revised Jun 2023.
    3. Hengxu Lin & Dong Zhou & Weiqing Liu & Jiang Bian, 2021. "Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport," Papers 2106.12950, arXiv.org, revised Jun 2021.
    4. Gabriel Borrageiro & Nick Firoozye & Paolo Barucca, 2021. "Reinforcement Learning for Systematic FX Trading," Papers 2110.04745, arXiv.org, revised May 2022.
    5. Miquel Noguer i Alonso & Sonam Srivastava, 2020. "Deep Reinforcement Learning for Asset Allocation in US Equities," Papers 2010.04404, arXiv.org.
    6. Robert Gk{e}barowski & Pawe{l} O'swik{e}cimka & Marcin Wk{a}torek & Stanis{l}aw Dro.zd.z, 2019. "Detecting correlations and triangular arbitrage opportunities in the Forex by means of multifractal detrended cross-correlations analysis," Papers 1906.07491, arXiv.org, revised Oct 2019.
    7. Yuanrong Wang & Yinsen Miao & Alexander CY Wong & Nikita P Granger & Christian Michler, 2023. "Domain-adapted Learning and Interpretability: DRL for Gas Trading," Papers 2301.08359, arXiv.org, revised Sep 2023.
    8. Jiezhu Cheng & Kaizhu Huang & Zibin Zheng, 2023. "Can Perturbations Help Reduce Investment Risks? Risk-Aware Stock Recommendation via Split Variational Adversarial Training," Papers 2304.11043, arXiv.org, revised Jan 2024.
    9. Hengxu Lin & Dong Zhou & Weiqing Liu & Jiang Bian, 2021. "Deep Risk Model: A Deep Learning Solution for Mining Latent Risk Factors to Improve Covariance Matrix Estimation," Papers 2107.05201, arXiv.org, revised Oct 2021.
    10. Xiao Yang & Weiqing Liu & Dong Zhou & Jiang Bian & Tie-Yan Liu, 2020. "Qlib: An AI-oriented Quantitative Investment Platform," Papers 2009.11189, arXiv.org.
    11. Jiexia Ye & Juanjuan Zhao & Kejiang Ye & Chengzhong Xu, 2020. "Multi-Graph Convolutional Network for Relationship-Driven Stock Movement Prediction," Papers 2005.04955, arXiv.org, revised Oct 2020.
    12. Abbas Haider & Hui Wang & Bryan Scotney & Glenn Hawe, 2022. "Predictive Market Making via Machine Learning," SN Operations Research Forum, Springer, vol. 3(1), pages 1-21, March.
    13. Tessa Bauman & Bruno Gav{s}perov & Stjepan Beguv{s}i'c & Zvonko Kostanjv{c}ar, 2023. "Deep Reinforcement Learning for Robust Goal-Based Wealth Management," Papers 2307.13501, arXiv.org.
    14. Ali Raheman & Anton Kolonin & Alexey Glushchenko & Arseniy Fokin & Ikram Ansari, 2022. "Adaptive Multi-Strategy Market-Making Agent For Volatile Markets," Papers 2204.13265, arXiv.org.
    15. Qianqian Xie & Weiguang Han & Yanzhao Lai & Min Peng & Jimin Huang, 2023. "The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges," Papers 2304.05351, arXiv.org, revised Apr 2023.
    16. Alejandra de la Rica Escudero & Eduardo C. Garrido-Merchan & Maria Coronado-Vaca, 2024. "Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent," Papers 2407.14486, arXiv.org.
    17. Hui Niu & Siyuan Li & Jian Li, 2022. "MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization," Papers 2210.01774, arXiv.org.
    18. Shwai He & Shi Gu, 2021. "Multi-modal Attention Network for Stock Movements Prediction," Papers 2112.13593, arXiv.org, revised Oct 2022.
    19. Ramit Sawhney & Shivam Agarwal & Vivek Mittal & Paolo Rosso & Vikram Nanda & Sudheer Chava, 2022. "Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models," Papers 2206.06320, arXiv.org.
    20. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay, 2020. "Bridging the gap between Markowitz planning and deep reinforcement learning," Papers 2010.09108, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2105.03844. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.