IDEAS home Printed from https://ideas.repec.org/a/gam/jijerp/v18y2021i19p10321-d647309.html
   My bibliography  Save this article

BiLSTM-I: A Deep Learning-Based Long Interval Gap-Filling Method for Meteorological Observation Data

Author

Listed:
  • Chuanjie Xie

    (State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
    These authors contributed equally to this work.)

  • Chong Huang

    (State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China
    These authors contributed equally to this work.)

  • Deqiang Zhang

    (Key Laboratory of Plant Resources Conservation and Sustainable Utilization, South China Botanical Garden, Chinese Academy of Sciences, Guangzhou 510650, China)

  • Wei He

    (State Key Laboratory of Resources and Environmental Information System, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China)

Abstract

Complete and high-resolution temperature observation data are important input parameters for agrometeorological disaster monitoring and ecosystem modelling. Due to the limitation of field meteorological observation conditions, observation data are commonly missing, and an appropriate data imputation method is necessary in meteorological data applications. In this paper, we focus on filling long gaps in meteorological observation data at field sites. A deep learning-based model, BiLSTM-I, is proposed to impute missing half-hourly temperature observations with high accuracy by considering temperature observations obtained manually at a low frequency. An encoder-decoder structure is adopted by BiLSTM-I, which is conducive to fully learning the potential distribution pattern of data. In addition, the BiLSTM-I model error function incorporates the difference between the final estimates and true observations. Therefore, the error function evaluates the imputation results more directly, and the model convergence error and the imputation accuracy are directly related, thus ensuring that the imputation error can be minimized at the time the model converges. The experimental analysis results show that the BiLSTM-I model designed in this paper is superior to other methods. For a test set with a time interval gap of 30 days, or a time interval gap of 60 days, the root mean square errors (RMSEs) remain stable, indicating the model’s excellent generalization ability for different missing value gaps. Although the model is only applied to temperature data imputation in this study, it also has the potential to be applied to other meteorological dataset-filling scenarios.

Suggested Citation

  • Chuanjie Xie & Chong Huang & Deqiang Zhang & Wei He, 2021. "BiLSTM-I: A Deep Learning-Based Long Interval Gap-Filling Method for Meteorological Observation Data," IJERPH, MDPI, vol. 18(19), pages 1-12, September.
  • Handle: RePEc:gam:jijerp:v:18:y:2021:i:19:p:10321-:d:647309
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1660-4601/18/19/10321/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1660-4601/18/19/10321/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Durbin, James & Koopman, Siem Jan, 2012. "Time Series Analysis by State Space Methods," OUP Catalogue, Oxford University Press, edition 2, number 9780199641178.
    2. Demirhan, Haydar & Renwick, Zoe, 2018. "Missing value imputation for short to mid-term horizontal solar irradiance data," Applied Energy, Elsevier, vol. 225(C), pages 998-1012.
    3. Carrizosa, Emilio & Olivares-Nadal, Alba V. & Ramírez-Cobo, Pepa, 2013. "Time series interpolation via global optimization of moments fitting," European Journal of Operational Research, Elsevier, vol. 230(1), pages 97-112.
    4. Hewamalage, Hansika & Bergmeir, Christoph & Bandara, Kasun, 2021. "Recurrent Neural Networks for Time Series Forecasting: Current status and future directions," International Journal of Forecasting, Elsevier, vol. 37(1), pages 388-427.
    5. Leonel Lara-Estrada & Livia Rasche & L. Enrique Sucar & Uwe A. Schneider, 2018. "Inferring Missing Climate Data for Agricultural Planning Using Bayesian Networks," Land, MDPI, vol. 7(1), pages 1-13, January.
    6. de Jong, Piet & Penzer, Jeremy, 2004. "The ARMA model in state space form," Statistics & Probability Letters, Elsevier, vol. 70(1), pages 119-125, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tommaso Proietti & Alessandra Luati, 2013. "Maximum likelihood estimation of time series models: the Kalman filter and beyond," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 15, pages 334-362, Edward Elgar Publishing.
    2. Pedregal, Diego J. & Trapero, Juan R., 2021. "Adjusted combination of moving averages: A forecasting system for medium-term solar irradiance," Applied Energy, Elsevier, vol. 298(C).
    3. Hang Qian, 2014. "A Flexible State Space Model And Its Applications," Journal of Time Series Analysis, Wiley Blackwell, vol. 35(2), pages 79-88, March.
    4. Tommaso Proietti, 2021. "Predictability, real time estimation, and the formulation of unobserved components models," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 433-454, April.
    5. Díaz, Guzmán & Moreno, Blanca & Coto, José & Gómez-Aleixandre, Javier, 2015. "Valuation of wind power distributed generation by using Longstaff–Schwartz option pricing method," Applied Energy, Elsevier, vol. 145(C), pages 223-233.
    6. Ledenyov, Dimitri O. & Ledenyov, Viktor O., 2013. "On the Stratonovich – Kalman - Bucy filtering algorithm application for accurate characterization of financial time series with use of state-space model by central banks," MPRA Paper 50235, University Library of Munich, Germany.
    7. Ledenyov, Dimitri O. & Ledenyov, Viktor O., 2015. "Wave function method to forecast foreign currencies exchange rates at ultra high frequency electronic trading in foreign currencies exchange markets," MPRA Paper 67470, University Library of Munich, Germany.
    8. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    9. Victor Bystrov, 2018. "Measuring the Natural Rates of Interest in Germany and Italy," Central European Journal of Economic Modelling and Econometrics, Central European Journal of Economic Modelling and Econometrics, vol. 10(4), pages 333-353, December.
    10. Yukai Yang & Luc Bauwens, 2018. "State-Space Models on the Stiefel Manifold with a New Approach to Nonlinear Filtering," Econometrics, MDPI, vol. 6(4), pages 1-22, December.
    11. Fernández-Macho, Javier, 2008. "Spectral estimation of a structural thin-plate smoothing model," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 189-195, September.
    12. Drew Creal & Siem Jan Koopman & Eric Zivot, 2008. "The Effect of the Great Moderation on the U.S. Business Cycle in a Time-varying Multivariate Trend-cycle Model," Tinbergen Institute Discussion Papers 08-069/4, Tinbergen Institute.
    13. Avanzi, Benjamin & Taylor, Greg & Vu, Phuong Anh & Wong, Bernard, 2020. "A multivariate evolutionary generalised linear model framework with adaptive estimation for claims reserving," Insurance: Mathematics and Economics, Elsevier, vol. 93(C), pages 50-71.
    14. François R. Velde, 2009. "Chronicle of a Deflation Unforetold," Journal of Political Economy, University of Chicago Press, vol. 117(4), pages 591-634, August.
    15. Chen, Peimin & Wu, Chunchi, 2014. "Default prediction with dynamic sectoral and macroeconomic frailties," Journal of Banking & Finance, Elsevier, vol. 40(C), pages 211-226.
    16. repec:zbw:bofitp:2019_008 is not listed on IDEAS
    17. Yue Zhao & Difang Wan, 2018. "Institutional high frequency trading and price discovery: Evidence from an emerging commodity futures market," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 38(2), pages 243-270, February.
    18. Wen Xu, 2016. "Estimation of Dynamic Panel Data Models with Stochastic Volatility Using Particle Filters," Econometrics, MDPI, vol. 4(4), pages 1-13, October.
    19. Scott Brave & R. Andrew Butters & Alejandro Justiniano, 2016. "Forecasting Economic Activity with Mixed Frequency Bayesian VARs," Working Paper Series WP-2016-5, Federal Reserve Bank of Chicago.
    20. repec:spo:wpmain:info:hdl:2441/1904 is not listed on IDEAS
    21. Hári, Norbert & De Waegenaere, Anja & Melenberg, Bertrand & Nijman, Theo E., 2008. "Estimating the term structure of mortality," Insurance: Mathematics and Economics, Elsevier, vol. 42(2), pages 492-504, April.
    22. Brave, Scott A. & Gascon, Charles & Kluender, William & Walstrum, Thomas, 2021. "Predicting benchmarked US state employment data in real time," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1261-1275.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijerp:v:18:y:2021:i:19:p:10321-:d:647309. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.