IDEAS home Printed from https://ideas.repec.org/p/ven/wpaper/201415.html
   My bibliography  Save this paper

Q-Learning-based financial trading systems with applications

Author

Listed:
  • Marco Corazza

    (Department of Economics, Ca� Foscari University of Venice; Advanced School of Economics in Venice.)

  • Francesco Bertoluzzo

    (Department of Economics, Ca� Foscari University of Venice.)

Abstract

The design of financial trading systems (FTSs) is a subject of high interest both for the academic environment and for the professional one due to the promises by machine learning methodologies. In this paper we consider the Reinforcement Learning-based policy evaluation approach known as Q-Learning algorithm (QLa). QLa is an algorithm which real-time optimizes its behavior in relation to the responses it gets from the environment in which it operates. In particular: first we introduce the essential aspects of QLa which are of interest for our purposes; second we present some original FTSs based on differently configured QLas; then we apply such FTSs to an artificial time series of daily stock prices and to six real ones from the Italian stock market belonging to the FTSE MIB basket. The results we achieve are generally satisfactory.

Suggested Citation

  • Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    • Handle: RePEc:ven:wpaper:2014:15
    as

    Download full text from publisher

    File URL: http://www.unive.it/pag/fileadmin/user_upload/dipartimenti/economia/doc/Pubblicazioni_scientifiche/working_papers/2014/WP_DSE_corazza_bertoluzzo_15_14.pdf
    File Function: First version, anno
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. J. Doyne Farmer, 2002. "Market force, ecology and evolution," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 11(5), pages 895-953, November.
    2. Bekiros, Stelios D., 2010. "Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach," Journal of Economic Dynamics and Control, Elsevier, vol. 34(6), pages 1153-1170, June.
    3. Francesco Bertoluzzo & Marco Corazza, 2006. "Financial trading systems: Is recurrent reinforcement the via?," Working Papers 141, Department of Applied Mathematics, Università Ca' Foscari Venezia.
    4. Xufre Casqueiro, Patricia & Rodrigues, Antonio J.L., 2006. "Neuro-dynamic trading methods," European Journal of Operational Research, Elsevier, vol. 175(3), pages 1400-1412, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.
    2. Taylan Kabbani & Ekrem Duman, 2022. "Deep Reinforcement Learning Approach for Trading Automation in The Stock Market," Papers 2208.07165, arXiv.org.
    3. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    2. Bekiros, Stelios D., 2015. "Heuristic learning in intraday trading under uncertainty," Journal of Empirical Finance, Elsevier, vol. 30(C), pages 34-49.
    3. Citera, Emanuele & Sau, Lino, 2019. "Complexity, Conventions and Instability: the role of monetary policy," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201924, University of Turin.
    4. Brock, William A. & Hommes, Cars H. & Wagener, Florian O. O., 2005. "Evolutionary dynamics in markets with many trader types," Journal of Mathematical Economics, Elsevier, vol. 41(1-2), pages 7-42, February.
    5. A. Corcos & J-P Eckmann & A. Malaspinas & Y. Malevergne & D. Sornette, 2002. "Imitation and contrarian behaviour: hyperbolic bubbles, crashes and chaos," Quantitative Finance, Taylor & Francis Journals, vol. 2(4), pages 264-281.
    6. Salzman, Diego & Trifan, Emanuela, 2006. "Emotions, Bayesian Inference, and Financial Decision Making," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 28163, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    7. Gaël Giraud & Céline Rochon, 2010. "Transition to Equilibrium in International Trades," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00657038, HAL.
    8. Getmansky, Mila & Lo, Andrew W. & Makarov, Igor, 2004. "An econometric model of serial correlation and illiquidity in hedge fund returns," Journal of Financial Economics, Elsevier, vol. 74(3), pages 529-609, December.
    9. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    10. Roberto Savona & Maxence Soumare & Jørgen Vitting Andersen, 2015. "Financial Symmetry and Moods in the Market," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-21, April.
    11. Jean-Philippe Bouchaud & Damien Challet, 2016. "Why have asset price properties changed so little in 200 years," Papers 1605.00634, arXiv.org.
    12. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    13. Westerhoff, Frank H., 2003. "Expectations driven distortions in the foreign exchange market," Journal of Economic Behavior & Organization, Elsevier, vol. 51(3), pages 389-412, July.
    14. Sornette, Didier & Zhou, Wei-Xing, 2006. "Importance of positive feedbacks and overconfidence in a self-fulfilling Ising model of financial markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(2), pages 704-726.
    15. Takumi Sueshige & Didier Sornette & Hideki Takayasu & Misako Takayasu, 2019. "Classification of position management strategies at the order-book level and their influences on future market-price formation," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-19, August.
    16. Omurtag, Ahmet & Sirovich, Lawrence, 2006. "Modeling a large population of traders: Mimesis and stability," Journal of Economic Behavior & Organization, Elsevier, vol. 61(4), pages 562-576, December.
    17. Maarten P. Scholl & Anisoara Calinescu & J. Doyne Farmer, 2021. "How market ecology explains market malfunction," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(26), pages 2015574118-, June.
    18. Adam Karp & Gary Van Vuuren, 2019. "Investment Implications Of The Fractal Market Hypothesis," Annals of Financial Economics (AFE), World Scientific Publishing Co. Pte. Ltd., vol. 14(01), pages 1-27, March.
    19. Feldman, Todd & Friedman, Daniel, 2008. "Humans, Robots and Market Crashes: A Laboratory Study ∗," Santa Cruz Department of Economics, Working Paper Series qt4kf382p6, Department of Economics, UC Santa Cruz.
    20. Li Lin & Didier Sornette, 2009. "Diagnostics of Rational Expectation Financial Bubbles with Stochastic Mean-Reverting Termination Times," Papers 0911.1921, arXiv.org.

    More about this item

    Keywords

    Financial trading system; Reinforcement Learning; Q-Learning algorithm; daily stock price time series; FTSE MIB basket.;
    All these keywords.

    JEL classification:

    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ven:wpaper:2014:15. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Geraldine Ludbrook (email available below). General contact details of provider: https://edirc.repec.org/data/dsvenit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.