IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2402.06638.html
   My bibliography  Save this paper

Transformers with Attentive Federated Aggregation for Time Series Stock Forecasting

Author

Listed:
  • Chu Myaet Thwal
  • Ye Lin Tun
  • Kitae Kim
  • Seong-Bae Park
  • Choong Seon Hong

Abstract

Recent innovations in transformers have shown their superior performance in natural language processing (NLP) and computer vision (CV). The ability to capture long-range dependencies and interactions in sequential data has also triggered a great interest in time series modeling, leading to the widespread use of transformers in many time series applications. However, being the most common and crucial application, the adaptation of transformers to time series forecasting has remained limited, with both promising and inconsistent results. In contrast to the challenges in NLP and CV, time series problems not only add the complexity of order or temporal dependence among input sequences but also consider trend, level, and seasonality information that much of this data is valuable for decision making. The conventional training scheme has shown deficiencies regarding model overfitting, data scarcity, and privacy issues when working with transformers for a forecasting task. In this work, we propose attentive federated transformers for time series stock forecasting with better performance while preserving the privacy of participating enterprises. Empirical results on various stock data from the Yahoo! Finance website indicate the superiority of our proposed scheme in dealing with the above challenges and data heterogeneity in federated learning.

Suggested Citation

  • Chu Myaet Thwal & Ye Lin Tun & Kitae Kim & Seong-Bae Park & Choong Seon Hong, 2024. "Transformers with Attentive Federated Aggregation for Time Series Stock Forecasting," Papers 2402.06638, arXiv.org.
  • Handle: RePEc:arx:papers:2402.06638
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2402.06638
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    2. Jiayu Qiu & Bin Wang & Changjun Zhou, 2020. "Forecasting stock prices with long-short term memory neural network based on attention mechanism," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-15, January.
    3. Sujin Pyo & Jaewook Lee & Mincheol Cha & Huisu Jang, 2017. "Predictability of machine learning techniques to forecast the trends of market index prices: Hypothesis testing for the Korean stock markets," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-17, November.
    4. Svetlana Borovkova & Ioannis Tsiamas, 2019. "An ensemble of LSTM neural networks for high‐frequency stock market classification," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 38(6), pages 600-619, September.
    5. Jia Wang & Tong Sun & Benyuan Liu & Yu Cao & Hongwei Zhu, 2021. "CLVSA: A Convolutional LSTM Based Variational Sequence-to-Sequence Model with Attention for Predicting Trends of Financial Markets," Papers 2104.04041, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Firuz Kamalov & Linda Smail & Ikhlaas Gurrib, 2021. "Stock price forecast with deep learning," Papers 2103.14081, arXiv.org.
    2. Labib Shami & Teddy Lazebnik, 2024. "Implementing Machine Learning Methods in Estimating the Size of the Non-observed Economy," Computational Economics, Springer;Society for Computational Economics, vol. 63(4), pages 1459-1476, April.
    3. Todd E. Clark & Florian Huber & Gary Koop & Massimiliano Marcellino, 2022. "Forecasting US Inflation Using Bayesian Nonparametric Models," Papers 2202.13793, arXiv.org.
    4. Tostes, Bernardo & Henriques, Sofia T. & Brockway, Paul E. & Heun, Matthew Kuperus & Domingos, Tiago & Sousa, Tânia, 2024. "On the right track? Energy use, carbon emissions, and intensities of world rail transportation, 1840–2020," Applied Energy, Elsevier, vol. 367(C).
    5. Janusz Gajda & Rafał Walasek, 2020. "Fractional differentiation and its use in machine learning," Working Papers 2020-32, Faculty of Economic Sciences, University of Warsaw.
    6. Lorenzo Menculini & Andrea Marini & Massimiliano Proietti & Alberto Garinei & Alessio Bozza & Cecilia Moretti & Marcello Marconi, 2021. "Comparing Prophet and Deep Learning to ARIMA in Forecasting Wholesale Food Prices," Forecasting, MDPI, vol. 3(3), pages 1-19, September.
    7. Todd E. Clark & Florian Huber & Gary Koop & Massimiliano Marcellino & Michael Pfarrhofer, 2023. "Tail Forecasting With Multivariate Bayesian Additive Regression Trees," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 64(3), pages 979-1022, August.
    8. Chen, Bin & Maung, Kenwin, 2023. "Time-varying forecast combination for high-dimensional data," Journal of Econometrics, Elsevier, vol. 237(2).
    9. Wang, Yijun & Andreeva, Galina & Martin-Barragan, Belen, 2023. "Machine learning approaches to forecasting cryptocurrency volatility: Considering internal and external determinants," International Review of Financial Analysis, Elsevier, vol. 90(C).
    10. Yuanrong Wang & Yinsen Miao & Alexander CY Wong & Nikita P Granger & Christian Michler, 2023. "Domain-adapted Learning and Interpretability: DRL for Gas Trading," Papers 2301.08359, arXiv.org, revised Sep 2023.
    11. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    12. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    13. Akash Doshi & Alexander Issa & Puneet Sachdeva & Sina Rafati & Somnath Rakshit, 2020. "Deep Stock Predictions," Papers 2006.04992, arXiv.org.
    14. Cardo-Miota, Javier & Trivedi, Rohit & Patra, Sandipan & Khadem, Shafi & Bahloul, Mohamed, 2024. "Data-driven approach for day-ahead System Non-Synchronous Penetration forecasting: A comprehensive framework, model development and analysis," Applied Energy, Elsevier, vol. 362(C).
    15. Stefenon, Stefano Frizzo & Seman, Laio Oriel & Aquino, Luiza Scapinello & Coelho, Leandro dos Santos, 2023. "Wavelet-Seq2Seq-LSTM with attention for time series forecasting of level of dams in hydroelectric power plants," Energy, Elsevier, vol. 274(C).
    16. Renigier-Biłozor Małgorzata & Janowski Artur, 2024. "Human-Machine Synergy in Real Estate Similarity Concept," Real Estate Management and Valuation, Sciendo, vol. 32(2), pages 13-30.
    17. Moreno, Sinvaldo Rodrigues & Seman, Laio Oriel & Stefenon, Stefano Frizzo & Coelho, Leandro dos Santos & Mariani, Viviana Cocco, 2024. "Enhancing wind speed forecasting through synergy of machine learning, singular spectral analysis, and variational mode decomposition," Energy, Elsevier, vol. 292(C).
    18. Subba Rao Polamuri & K. Srinnivas & A. Krishna Mohan, 2023. "Prediction of stock price growth for novel greedy heuristic optimized multi-instances quantitative (NGHOMQ)," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 14(1), pages 353-366, February.
    19. Mojtaba Sedighi & Hossein Jahangirnia & Mohsen Gharakhani & Saeed Farahani Fard, 2019. "A Novel Hybrid Model for Stock Price Forecasting Based on Metaheuristics and Support Vector Machine," Data, MDPI, vol. 4(2), pages 1-28, May.
    20. Firuz Kamalov, 2019. "Forecasting significant stock price changes using neural networks," Papers 1912.08791, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2402.06638. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.