IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2101.03867.html
   My bibliography  Save this paper

A Reinforcement Learning Based Encoder-Decoder Framework for Learning Stock Trading Rules

Author

Listed:
  • Mehran Taghian
  • Ahmad Asadi
  • Reza Safabakhsh

Abstract

A wide variety of deep reinforcement learning (DRL) models have recently been proposed to learn profitable investment strategies. The rules learned by these models outperform the previous strategies specially in high frequency trading environments. However, it is shown that the quality of the extracted features from a long-term sequence of raw prices of the instruments greatly affects the performance of the trading rules learned by these models. Employing a neural encoder-decoder structure to extract informative features from complex input time-series has proved very effective in other popular tasks like neural machine translation and video captioning in which the models face a similar problem. The encoder-decoder framework extracts highly informative features from a long sequence of prices along with learning how to generate outputs based on the extracted features. In this paper, a novel end-to-end model based on the neural encoder-decoder framework combined with DRL is proposed to learn single instrument trading strategies from a long sequence of raw prices of the instrument. The proposed model consists of an encoder which is a neural structure responsible for learning informative features from the input sequence, and a decoder which is a DRL model responsible for learning profitable strategies based on the features extracted by the encoder. The parameters of the encoder and the decoder structures are learned jointly, which enables the encoder to extract features fitted to the task of the decoder DRL. In addition, the effects of different structures for the encoder and various forms of the input sequences on the performance of the learned strategies are investigated. Experimental results showed that the proposed model outperforms other state-of-the-art models in highly dynamic environments.

Suggested Citation

  • Mehran Taghian & Ahmad Asadi & Reza Safabakhsh, 2021. "A Reinforcement Learning Based Encoder-Decoder Framework for Learning Stock Trading Rules," Papers 2101.03867, arXiv.org.
  • Handle: RePEc:arx:papers:2101.03867
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2101.03867
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Gomber & Martin Haferkorn, 2013. "High-Frequency-Trading," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 5(2), pages 97-99, April.
    2. Matthew F. Dixon & Nicholas G. Polson & Vadim O. Sokolov, 2019. "Deep learning for spatio‐temporal modeling: Dynamic traffic flows and high frequency trading," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 35(3), pages 788-807, May.
    3. Conrad, Jennifer & Wahal, Sunil & Xiang, Jin, 2015. "High-frequency quoting, trading, and the efficiency of prices," Journal of Financial Economics, Elsevier, vol. 116(2), pages 271-291.
    4. Jingyuan Wang & Yang Zhang & Ke Tang & Junjie Wu & Zhang Xiong, 2019. "AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks," Papers 1908.02646, arXiv.org.
    5. Luo, Suyuan & Lin, Xudong & Zheng, Zunxin, 2019. "A novel CNN-DDPG based AI-trader: Performance and roles in business operations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 131(C), pages 68-79.
    6. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    7. Allen, Franklin & Karjalainen, Risto, 1999. "Using genetic algorithms to find technical trading rules," Journal of Financial Economics, Elsevier, vol. 51(2), pages 245-271, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuling Huang & Kai Cui & Yunlin Song & Zongren Chen, 2023. "A Multi-Scaling Reinforcement Learning Trading System Based on Multi-Scaling Convolutional Neural Networks," Mathematics, MDPI, vol. 11(11), pages 1-19, May.
    2. Gang Hu, 2023. "Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models," Papers 2311.05743, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wei Pan & Jide Li & Xiaoqiang Li, 2020. "Portfolio Learning Based on Deep Learning," Future Internet, MDPI, vol. 12(11), pages 1-13, November.
    2. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    3. Fabrice Rousseau & Herve Boco & Laurent Germain, 2020. "High Frequency Trading: Strategic Competition Between Slow and Fast Traders," Economics Department Working Paper Series n296-20.pdf, Department of Economics, National University of Ireland - Maynooth.
    4. Tidor-Vlad Pricope, 2021. "Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review," Papers 2106.00123, arXiv.org.
    5. Alfonso-Sánchez, Sherly & Solano, Jesús & Correa-Bahnsen, Alejandro & Sendova, Kristina P. & Bravo, Cristián, 2024. "Optimizing credit limit adjustments under adversarial goals using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 315(2), pages 802-817.
    6. Li, Jiawen & Yu, Tao & Yang, Bo, 2021. "A data-driven output voltage control of solid oxide fuel cell using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 304(C).
    7. Andriosopoulos, Kostas & Nomikos, Nikos, 2014. "Performance replication of the Spot Energy Index with optimal equity portfolio selection: Evidence from the UK, US and Brazilian markets," European Journal of Operational Research, Elsevier, vol. 234(2), pages 571-582.
    8. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    9. Chia-Lin Chang & Jukka Ilomäki & Hannu Laurila & Michael McAleer, 2018. "Long Run Returns Predictability and Volatility with Moving Averages," Risks, MDPI, vol. 6(4), pages 1-18, September.
    10. Pereira, Robert, 1999. "Forecasting Ability But No Profitability: An Empirical Evaluation of Genetic Algorithm-optimised Technical Trading Rules," MPRA Paper 9055, University Library of Munich, Germany.
    11. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    12. Indriawan, Ivan & Martinez, Valeria & Tse, Yiuman, 2021. "The impact of the change in USDA announcement release procedures on agricultural commodity futures," Journal of Commodity Markets, Elsevier, vol. 23(C).
    13. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    14. Victor Elsa Sapphira & Razali Muhammad Najib & Ali Hishamuddin Mohd., 2023. "The Dynamics of the Impact of the Covid-19 Pandemic on Pan-Asia’s Real Estate Investment Trusts," Real Estate Management and Valuation, Sciendo, vol. 31(4), pages 11-22, December.
    15. Imen Azzouz & Wiem Fekih Hassen, 2023. "Optimization of Electric Vehicles Charging Scheduling Based on Deep Reinforcement Learning: A Decentralized Approach," Energies, MDPI, vol. 16(24), pages 1-18, December.
    16. Jacob W. Crandall & Mayada Oudah & Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael A. Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," TSE Working Papers 17-806, Toulouse School of Economics (TSE).
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," IAST Working Papers 17-68, Institute for Advanced Study in Toulouse (IAST).
      • Jacob Crandall & Mayada Oudah & Fatimah Ishowo-Oloko Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Post-Print hal-01897802, HAL.
    17. Sun, Alexander Y., 2020. "Optimal carbon storage reservoir management through deep reinforcement learning," Applied Energy, Elsevier, vol. 278(C).
    18. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    19. Serdengeçti, Süleyman & Sensoy, Ahmet & Nguyen, Duc Khuong, 2021. "Dynamics of return and liquidity (co) jumps in emerging foreign exchange markets," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 73(C).
    20. Shrutika Mishra & A. R. Tripathi, 2021. "AI business model: an integrative business approach," Journal of Innovation and Entrepreneurship, Springer, vol. 10(1), pages 1-21, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2101.03867. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.