IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i6p1503-d1102009.html
   My bibliography  Save this article

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

Author

Listed:
  • Yu-Ting Bai

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China
    Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing 100048, China)

  • Wei Jia

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Xue-Bo Jin

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China
    Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing 100048, China)

  • Ting-Li Su

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Jian-Lei Kong

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

  • Zhi-Gang Shi

    (School of Artificial Intelligence, Beijing Technology and Business University, Beijing 100048, China)

Abstract

The predictions from time series data can help us sense development trends and make scientific decisions in advance. The commonly used forecasting methods with backpropagation consume a lot of computational resources. The deep echo state network (DeepESN) is an advanced prediction method with a deep neural network structure and training algorithm without backpropagation. In this paper, a Bayesian optimization algorithm (BOA) is proposed to optimize DeepESN to address the problem of increasing parameter scale. Firstly, the DeepESN was studied and constructed as the basic prediction model for the time series data. Secondly, the BOA was reconstructed, based on the DeepESN, for optimal parameter searching. The algorithm is proposed within the framework of the DeepESN. Thirdly, an experiment was conducted to verify the DeepESN with a BOA within three datasets: simulation data generated from computer programs, a real humidity dataset collected from Beijing, and a power load dataset obtained from America. Compared with the models of BP (backpropagation), LSTM (long short-term memory), GRU (gated recurrent unit), and ESN (echo state network), DeepESN obtained optimal results, which were 0.0719, 18.6707, and 764.5281 using RMSE evaluation. While getting better accuracy, the BOA optimization time was only 323.4 s, 563.2 s, and 9854 s for the three datasets. It is more efficient than grid search and grey wolf optimizer.

Suggested Citation

  • Yu-Ting Bai & Wei Jia & Xue-Bo Jin & Ting-Li Su & Jian-Lei Kong & Zhi-Gang Shi, 2023. "Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization," Mathematics, MDPI, vol. 11(6), pages 1-22, March.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:6:p:1503-:d:1102009
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/6/1503/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/6/1503/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tim Leung & Theodore Zhao, 2021. "Financial Time Series Analysis and Forecasting with HHT Feature Generation and Machine Learning," Papers 2105.10871, arXiv.org.
    2. Wei Bao & Jun Yue & Yulei Rao, 2017. "A deep learning framework for financial time series using stacked autoencoders and long-short term memory," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-24, July.
    3. Xue-Bo Jin & Zhong-Yao Wang & Wen-Tao Gong & Jian-Lei Kong & Yu-Ting Bai & Ting-Li Su & Hui-Jun Ma & Prasun Chakrabarti, 2023. "Variational Bayesian Network with Information Interpretability Filtering for Air Quality Forecasting," Mathematics, MDPI, vol. 11(4), pages 1-18, February.
    4. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    5. Patrick L. McDermott & Christopher K. Wikle, 2019. "Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting," Environmetrics, John Wiley & Sons, Ltd., vol. 30(3), May.
    6. Xin Xu & Cheng-Cai Yang & Yang Xiao & Jian-Lei Kong, 2023. "A Fine-Grained Recognition Neural Network with High-Order Feature Maps via Graph-Based Embedding for Natural Bird Diversity Conservation," IJERPH, MDPI, vol. 20(6), pages 1-20, March.
    7. Yuebing Xu & Jing Zhang & Zuqiang Long & Yan Chen, 2018. "A Novel Dual-Scale Deep Belief Network Method for Daily Urban Water Demand Forecasting," Energies, MDPI, vol. 11(5), pages 1-15, April.
    8. Jessica Wojtkiewicz & Matin Hosseini & Raju Gottumukkala & Terrence Lynn Chambers, 2019. "Hour-Ahead Solar Irradiance Forecasting Using Multivariate Gated Recurrent Units," Energies, MDPI, vol. 12(21), pages 1-13, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yan Gao & Baifu Cao & Wenhao Yu & Lu Yi & Fengqi Guo, 2024. "Short-Term Wind Speed Prediction for Bridge Site Area Based on Wavelet Denoising OOA-Transformer," Mathematics, MDPI, vol. 12(12), pages 1-22, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nadja Klein & Michael Stanley Smith & David J. Nott, 2020. "Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices," Papers 2010.01844, arXiv.org, revised May 2021.
    2. Md. Nazmul Hasan & Rafia Nishat Toma & Abdullah-Al Nahid & M M Manjurul Islam & Jong-Myon Kim, 2019. "Electricity Theft Detection in Smart Grid Systems: A CNN-LSTM Based Approach," Energies, MDPI, vol. 12(17), pages 1-18, August.
    3. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    4. Jaydip Sen & Sidra Mehtab & Abhishek Dutta & Saikat Mondal, 2022. "Precise Stock Price Prediction for Optimized Portfolio Design Using an LSTM Model," Papers 2203.01326, arXiv.org.
    5. Jaydip Sen & Sidra Mehtab, 2021. "Design and Analysis of Robust Deep Learning Models for Stock Price Prediction," Papers 2106.09664, arXiv.org.
    6. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    7. James Wallbridge, 2020. "Transformers for Limit Order Books," Papers 2003.00130, arXiv.org.
    8. Antoine Proteau & Antoine Tahan & Ryad Zemouri & Marc Thomas, 2023. "Predicting the quality of a machined workpiece with a variational autoencoder approach," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 719-737, February.
    9. Yang Qiao & Yiping Xia & Xiang Li & Zheng Li & Yan Ge, 2023. "Higher-order Graph Attention Network for Stock Selection with Joint Analysis," Papers 2306.15526, arXiv.org.
    10. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2018. "DeepLOB: Deep Convolutional Neural Networks for Limit Order Books," Papers 1808.03668, arXiv.org, revised Jan 2020.
    11. Daiki Matsunaga & Toyotaro Suzumura & Toshihiro Takahashi, 2019. "Exploring Graph Neural Networks for Stock Market Predictions with Rolling Window Analysis," Papers 1909.10660, arXiv.org, revised Nov 2019.
    12. Emil Kraft & Dogan Keles & Wolf Fichtner, 2020. "Modeling of frequency containment reserve prices with econometrics and artificial intelligence," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(8), pages 1179-1197, December.
    13. Bokde, Neeraj Dhanraj & Tranberg, Bo & Andresen, Gorm Bruun, 2021. "Short-term CO2 emissions forecasting based on decomposition approaches and its impact on electricity market scheduling," Applied Energy, Elsevier, vol. 281(C).
    14. Raphael Paulo Beal Piovezan & Pedro Paulo Andrade Junior & Sérgio Luciano Ávila, 2024. "Machine Learning Method for Return Direction Forecast of Exchange Traded Funds (ETFs) Using Classification and Regression Models," Computational Economics, Springer;Society for Computational Economics, vol. 63(5), pages 1827-1852, May.
    15. Hakan Pabuccu & Adrian Barbu, 2023. "Feature Selection with Annealing for Forecasting Financial Time Series," Papers 2303.02223, arXiv.org, revised Feb 2024.
    16. Daniel Manfre Jaimes & Manuel Zamudio López & Hamidreza Zareipour & Mike Quashie, 2023. "A Hybrid Model for Multi-Day-Ahead Electricity Price Forecasting considering Price Spikes," Forecasting, MDPI, vol. 5(3), pages 1-23, July.
    17. Vu, Ba Hau & Chung, Il-Yop, 2022. "Optimal generation scheduling and operating reserve management for PV generation using RNN-based forecasting models for stand-alone microgrids," Renewable Energy, Elsevier, vol. 195(C), pages 1137-1154.
    18. Murat Aydogdu & Hakan Saraoglu & David Louton, 2019. "Using long short‐term memory neural networks to analyze SEC 13D filings: A recipe for human and machine interaction," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 26(4), pages 153-163, October.
    19. JoonBum Leem & Ha Young Kim, 2020. "Action-specialized expert ensemble trading system with extended discrete action space using deep reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-39, July.
    20. Zhaofeng Zhang & Banghao Chen & Shengxin Zhu & Nicolas Langren'e, 2024. "Quantformer: from attention to profit with a quantitative transformer trading strategy," Papers 2404.00424, arXiv.org, revised Oct 2024.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:6:p:1503-:d:1102009. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.