IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2002.11523.html
   My bibliography  Save this paper

Using Reinforcement Learning in the Algorithmic Trading Problem

Author

Listed:
  • Evgeny Ponomarev
  • Ivan Oseledets
  • Andrzej Cichocki

Abstract

The development of reinforced learning methods has extended application to many areas including algorithmic trading. In this paper trading on the stock exchange is interpreted into a game with a Markov property consisting of states, actions, and rewards. A system for trading the fixed volume of a financial instrument is proposed and experimentally tested; this is based on the asynchronous advantage actor-critic method with the use of several neural network architectures. The application of recurrent layers in this approach is investigated. The experiments were performed on real anonymized data. The best architecture demonstrated a trading strategy for the RTS Index futures (MOEX:RTSI) with a profitability of 66% per annum accounting for commission. The project source code is available via the following link: http://github.com/evgps/a3c_trading.

Suggested Citation

  • Evgeny Ponomarev & Ivan Oseledets & Andrzej Cichocki, 2020. "Using Reinforcement Learning in the Algorithmic Trading Problem," Papers 2002.11523, arXiv.org.
    • Handle: RePEc:arx:papers:2002.11523
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2002.11523
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bekiros, Stelios D., 2010. "Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach," Journal of Economic Dynamics and Control, Elsevier, vol. 34(6), pages 1153-1170, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    2. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    3. Amirhosein Mosavi & Yaser Faghan & Pedram Ghamisi & Puhong Duan & Sina Faizollahzadeh Ardabili & Ely Salwana & Shahab S. Band, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Mathematics, MDPI, vol. 8(10), pages 1-42, September.
    4. Hommes, Cars, 2011. "The heterogeneous expectations hypothesis: Some evidence from the lab," Journal of Economic Dynamics and Control, Elsevier, vol. 35(1), pages 1-24, January.
    5. Bekiros, Stelios & Marcellino, Massimiliano, 2013. "The multiscale causal dynamics of foreign exchange markets," Journal of International Money and Finance, Elsevier, vol. 33(C), pages 282-305.
    6. Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    7. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Deep Reinforcement Learning for Trading," Papers 1911.10107, arXiv.org.
    8. Cars Hommes, 2010. "The heterogeneous expectations hypothesis: some evidence from the lab," Post-Print hal-00753041, HAL.
    9. Bekiros, Stelios D., 2015. "Heuristic learning in intraday trading under uncertainty," Journal of Empirical Finance, Elsevier, vol. 30(C), pages 34-49.
    10. Stelios Bekiros, 2014. "Detecting nonlinear dependencies in foreign exchange markets: A multistep filtering approach," Working Papers 2014-182, Department of Research, Ipag Business School.
    11. Gradojevic, Nikola & Gençay, Ramazan, 2013. "Fuzzy logic, trading uncertainty and technical trading," Journal of Banking & Finance, Elsevier, vol. 37(2), pages 578-586.
    12. Chronopoulos, Dimitris K. & Papadimitriou, Fotios I. & Vlastakis, Nikolaos, 2018. "Information demand and stock return predictability," Journal of International Money and Finance, Elsevier, vol. 80(C), pages 59-74.
    13. Hassanniakalager, Arman & Sermpinis, Georgios & Stasinakis, Charalampos, 2021. "Trading the foreign exchange market with technical analysis and Bayesian Statistics," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 230-251.
    14. Konstandinos Chourmouziadis & Dimitra K. Chourmouziadou & Prodromos D. Chatzoglou, 2021. "Embedding Four Medium-Term Technical Indicators to an Intelligent Stock Trading Fuzzy System for Predicting: A Portfolio Management Approach," Computational Economics, Springer;Society for Computational Economics, vol. 57(4), pages 1183-1216, April.
    15. Schnaubelt, Matthias & Fischer, Thomas G. & Krauss, Christopher, 2020. "Separating the signal from the noise – Financial machine learning for Twitter," Journal of Economic Dynamics and Control, Elsevier, vol. 114(C).
    16. Zhong, Li-Xin & Xu, Wen-Juan & Chen, Rong-Da & He, Yun-Xin & Qiu, Tian & Ren, Fei & Shi, Yong-Dong & Zhong, Chen-Yang, 2020. "Multiple learning mechanisms promote cooperation in public goods games with project selection," Chaos, Solitons & Fractals, Elsevier, vol. 133(C).
    17. Soufian, Mona & Forbes, William & Hudson, Robert, 2014. "Adapting financial rationality: Is a new paradigm emerging?," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 25(8), pages 724-742.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2002.11523. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.