IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v34y2010i6p1153-1170.html
   My bibliography  Save this article

Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach

Author

Listed:
  • Bekiros, Stelios D.

Abstract

The present study addresses the learning mechanism of boundedly rational agents in the dynamic and noisy environment of financial markets. The main objective is the development of a system that "decodes" the knowledge-acquisition strategy and the decision-making process of technical analysts called "chartists". It advances the literature on heterogeneous learning in speculative markets by introducing a trading system wherein market environment and agent beliefs are represented by fuzzy inference rules. The resulting functionality leads to the derivation of the parameters of the fuzzy rules by means of adaptive training. In technical terms, it expands the literature that has utilized Actor-Critic reinforcement learning and fuzzy systems in agent-based applications, by presenting an adaptive fuzzy reinforcement learning approach that provides with accurate and prompt identification of market turning points and thus higher predictability. The purpose of this paper is to illustrate this concretely through a comparative investigation against other well-established models. The results indicate that with the inclusion of transaction costs, the profitability of the novel system in case of NASDAQ Composite, FTSE100 and NIKKEI255 indices is consistently superior to that of a Recurrent Neural Network, a Markov-switching model and a Buy and Hold strategy. Overall, the proposed system via the reinforcement learning mechanism, the fuzzy rule-based state space modeling and the adaptive action selection policy, leads to superior predictions upon the direction-of-change of the market.

Suggested Citation

  • Bekiros, Stelios D., 2010. "Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach," Journal of Economic Dynamics and Control, Elsevier, vol. 34(6), pages 1153-1170, June.
  • Handle: RePEc:eee:dyncon:v:34:y:2010:i:6:p:1153-1170
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165-1889(10)00016-3
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Henriksson, Roy D & Merton, Robert C, 1981. "On Market Timing and Investment Performance. II. Statistical Procedures for Evaluating Forecasting Skills," The Journal of Business, University of Chicago Press, vol. 54(4), pages 513-533, October.
    2. repec:bla:jfinan:v:55:y:2000:i:4:p:1705-1770 is not listed on IDEAS
    3. Tay, Nicholas S. P. & Linn, Scott C., 2001. "Fuzzy inductive reasoning, expectation formation and the behavior of security prices," Journal of Economic Dynamics and Control, Elsevier, vol. 25(3-4), pages 321-361, March.
    4. Shleifer, Andrei & Summers, Lawrence H, 1990. "The Noise Trader Approach to Finance," Journal of Economic Perspectives, American Economic Association, vol. 4(2), pages 19-33, Spring.
    5. William Schwert, G., 2002. "Stock volatility in the new millennium: how wacky is Nasdaq?," Journal of Monetary Economics, Elsevier, vol. 49(1), pages 3-26, January.
    6. Cheol‐Ho Park & Scott H. Irwin, 2007. "What Do We Know About The Profitability Of Technical Analysis?," Journal of Economic Surveys, Wiley Blackwell, vol. 21(4), pages 786-826, September.
    7. Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
    8. La Porta, Rafael, et al, 1997. "Good News for Value Stocks: Further Evidence on Market Efficiency," Journal of Finance, American Finance Association, vol. 52(2), pages 859-874, June.
    9. Gradojevic, Nikola, 2007. "Non-linear, hybrid exchange rate modeling and trading profitability in the foreign exchange market," Journal of Economic Dynamics and Control, Elsevier, vol. 31(2), pages 557-574, February.
    10. Bekaert, Geert & Wu, Guojun, 2000. "Asymmetric Volatility and Risk in Equity Markets," The Review of Financial Studies, Society for Financial Studies, vol. 13(1), pages 1-42.
    11. Franses,Philip Hans & Dijk,Dick van, 2000. "Non-Linear Time Series Models in Empirical Finance," Cambridge Books, Cambridge University Press, number 9780521770415, September.
    12. Gencay, Ramazan, 1998. "Optimization of technical trading strategies and the profitability in security markets," Economics Letters, Elsevier, vol. 59(2), pages 249-254, May.
    13. Robert J. Shiller, 2003. "From Efficient Markets Theory to Behavioral Finance," Journal of Economic Perspectives, American Economic Association, vol. 17(1), pages 83-104, Winter.
    14. G. Wenchi Kao & Christopher K. Ma, 1992. "Memories, heteroscedasticity, and price limit in Currency futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 12(6), pages 679-692, December.
    15. Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. "On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
    16. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    17. Po-Hsuan Hsu & Chung-Ming Kuan, 2005. "Reexamining the Profitability of Technical Analysis with Data Snooping Checks," Journal of Financial Econometrics, Oxford University Press, vol. 3(4), pages 606-628.
    18. Fama, Eugene F & French, Kenneth R, 1995. "Size and Book-to-Market Factors in Earnings and Returns," Journal of Finance, American Finance Association, vol. 50(1), pages 131-155, March.
    19. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    20. Fama, Eugene F, 1991. "Efficient Capital Markets: II," Journal of Finance, American Finance Association, vol. 46(5), pages 1575-1617, December.
    21. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    22. Lawrence, Michael & O'Connor, Marcus, 1992. "Exploring judgemental forecasting," International Journal of Forecasting, Elsevier, vol. 8(1), pages 15-26, June.
    23. Levich, Richard M. & Thomas, Lee III, 1993. "The significance of technical trading-rule profits in the foreign exchange market: a bootstrap approach," Journal of International Money and Finance, Elsevier, vol. 12(5), pages 451-474, October.
    24. Fernandez-Rodriguez, Fernando & Gonzalez-Martel, Christian & Sosvilla-Rivero, Simon, 2000. "On the profitability of technical trading rules based on artificial neural networks:: Evidence from the Madrid stock market," Economics Letters, Elsevier, vol. 69(1), pages 89-94, October.
    25. Christie, Andrew A., 1982. "The stochastic behavior of common stock variances : Value, leverage and interest rate effects," Journal of Financial Economics, Elsevier, vol. 10(4), pages 407-432, December.
    26. Andrew W. Lo & Harry Mamaysky & Jiang Wang, 2000. "Foundations of Technical Analysis: Computational Algorithms, Statistical Inference, and Empirical Implementation," Journal of Finance, American Finance Association, vol. 55(4), pages 1705-1765, August.
    27. Paul R. Krugman, 1987. "Trigger Strategies and Price Dynamics in Equity and Foreign Exchange Markets," NBER Working Papers 2459, National Bureau of Economic Research, Inc.
    28. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    29. Teo Jasic & Douglas Wood, 2004. "The profitability of daily stock market indices trades based on neural network predictions: case study for the S&P 500, the DAX, the TOPIX and the FTSE in the period 1965-1999," Applied Financial Economics, Taylor & Francis Journals, vol. 14(4), pages 285-297.
    30. Black, Fischer, 1986. "Noise," Journal of Finance, American Finance Association, vol. 41(3), pages 529-543, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hassanniakalager, Arman & Sermpinis, Georgios & Stasinakis, Charalampos, 2021. "Trading the foreign exchange market with technical analysis and Bayesian Statistics," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 230-251.
    2. Evgeny Ponomarev & Ivan Oseledets & Andrzej Cichocki, 2020. "Using Reinforcement Learning in the Algorithmic Trading Problem," Papers 2002.11523, arXiv.org.
    3. Bekiros, Stelios D., 2015. "Heuristic learning in intraday trading under uncertainty," Journal of Empirical Finance, Elsevier, vol. 30(C), pages 34-49.
    4. Hommes, Cars, 2011. "The heterogeneous expectations hypothesis: Some evidence from the lab," Journal of Economic Dynamics and Control, Elsevier, vol. 35(1), pages 1-24, January.
    5. Bekiros, Stelios & Marcellino, Massimiliano, 2013. "The multiscale causal dynamics of foreign exchange markets," Journal of International Money and Finance, Elsevier, vol. 33(C), pages 282-305.
    6. Gradojevic, Nikola & Gençay, Ramazan, 2013. "Fuzzy logic, trading uncertainty and technical trading," Journal of Banking & Finance, Elsevier, vol. 37(2), pages 578-586.
    7. Cars Hommes, 2010. "The heterogeneous expectations hypothesis: some evidence from the lab," Post-Print hal-00753041, HAL.
    8. Schnaubelt, Matthias & Fischer, Thomas G. & Krauss, Christopher, 2020. "Separating the signal from the noise – Financial machine learning for Twitter," Journal of Economic Dynamics and Control, Elsevier, vol. 114(C).
    9. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    10. Zhong, Li-Xin & Xu, Wen-Juan & Chen, Rong-Da & He, Yun-Xin & Qiu, Tian & Ren, Fei & Shi, Yong-Dong & Zhong, Chen-Yang, 2020. "Multiple learning mechanisms promote cooperation in public goods games with project selection," Chaos, Solitons & Fractals, Elsevier, vol. 133(C).
    11. Chronopoulos, Dimitris K. & Papadimitriou, Fotios I. & Vlastakis, Nikolaos, 2018. "Information demand and stock return predictability," Journal of International Money and Finance, Elsevier, vol. 80(C), pages 59-74.
    12. Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    13. Soufian, Mona & Forbes, William & Hudson, Robert, 2014. "Adapting financial rationality: Is a new paradigm emerging?," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 25(8), pages 724-742.
    14. Konstandinos Chourmouziadis & Dimitra K. Chourmouziadou & Prodromos D. Chatzoglou, 2021. "Embedding Four Medium-Term Technical Indicators to an Intelligent Stock Trading Fuzzy System for Predicting: A Portfolio Management Approach," Computational Economics, Springer;Society for Computational Economics, vol. 57(4), pages 1183-1216, April.
    15. Stelios Bekiros, 2014. "Detecting nonlinear dependencies in foreign exchange markets: A multistep filtering approach," Working Papers 2014-182, Department of Research, Ipag Business School.
    16. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Deep Reinforcement Learning for Trading," Papers 1911.10107, arXiv.org.
    17. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    18. Amirhosein Mosavi & Yaser Faghan & Pedram Ghamisi & Puhong Duan & Sina Faizollahzadeh Ardabili & Ely Salwana & Shahab S. Band, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Mathematics, MDPI, vol. 8(10), pages 1-42, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stelios D. Bekiros, 2013. "Irrational fads, short‐term memory emulation, and asset predictability," Review of Financial Economics, John Wiley & Sons, vol. 22(4), pages 213-219, November.
    2. Bekiros, Stelios D., 2015. "Heuristic learning in intraday trading under uncertainty," Journal of Empirical Finance, Elsevier, vol. 30(C), pages 34-49.
    3. Bekiros, Stelios D., 2010. "Fuzzy adaptive decision-making for boundedly rational traders in speculative stock markets," European Journal of Operational Research, Elsevier, vol. 202(1), pages 285-293, April.
    4. S. D. Bekiros & D. A. Georgoutsos, 2008. "Direction-of-change forecasting using a volatility-based recurrent neural network," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(5), pages 407-417.
    5. Razvan Stefanescu & Ramona Dumitriu, 2016. "Contrarian and Momentum Profits during Periods of High Trading Volume preceded by Stock Prices Shocks," Risk in Contemporary Economy, "Dunarea de Jos" University of Galati, Faculty of Economics and Business Administration, pages 378-384.
    6. Stelios Bekiros & Dimitris Georgoutsos, 2008. "Non-linear dynamics in financial asset returns: the predictive power of the CBOE volatility index," The European Journal of Finance, Taylor & Francis Journals, vol. 14(5), pages 397-408.
    7. Cheol‐Ho Park & Scott H. Irwin, 2007. "What Do We Know About The Profitability Of Technical Analysis?," Journal of Economic Surveys, Wiley Blackwell, vol. 21(4), pages 786-826, September.
    8. Stephan Schulmeister, 2000. "Technical Analysis and Exchange Rate Dynamics," WIFO Studies, WIFO, number 25857, March.
    9. Lukas Menkhoff & Mark P. Taylor, 2007. "The Obstinate Passion of Foreign Exchange Professionals: Technical Analysis," Journal of Economic Literature, American Economic Association, vol. 45(4), pages 936-972, December.
    10. Chiarella, Carl & He, Xue-Zhong & Hommes, Cars, 2006. "A dynamic analysis of moving average rules," Journal of Economic Dynamics and Control, Elsevier, vol. 30(9-10), pages 1729-1753.
    11. He, Xue-Zhong & Zheng, Min, 2010. "Dynamics of moving average rules in a continuous-time financial market model," Journal of Economic Behavior & Organization, Elsevier, vol. 76(3), pages 615-634, December.
    12. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2005. "Volatility forecasting," CFS Working Paper Series 2005/08, Center for Financial Studies (CFS).
    13. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    14. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    15. Ender Su & John Bilson, 2011. "Trading asymmetric trend and volatility by leverage trend GARCH in Taiwan stock index," Applied Economics, Taylor & Francis Journals, vol. 43(26), pages 3891-3905.
    16. Madhusudan Karmakar, 2007. "Asymmetric Volatility and Risk-return Relationship in the Indian Stock Market," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 8(1), pages 99-116, January.
    17. Peter Christoffersen & Francis X. Diebold, 2002. "Financial Asset Returns, Market Timing, and Volatility Dynamics," CIRANO Working Papers 2002s-02, CIRANO.
    18. Qianwei Ying & Tahir Yousaf & Qurat ul Ain & Yasmeen Akhtar & Muhammad Shahid Rasheed, 2019. "Stock Investment and Excess Returns: A Critical Review in the Light of the Efficient Market Hypothesis," JRFM, MDPI, vol. 12(2), pages 1-22, June.
    19. Peter F. Christoffersen & Francis X. Diebold, 2006. "Financial Asset Returns, Direction-of-Change Forecasting, and Volatility Dynamics," Management Science, INFORMS, vol. 52(8), pages 1273-1287, August.
    20. Lee, Chun I & Gleason, Kimberly C. & Mathur, Ike, 2001. "Trading rule profits in Latin American currency spot rates," International Review of Financial Analysis, Elsevier, vol. 10(2), pages 135-156.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:34:y:2010:i:6:p:1153-1170. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.