IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v225y2018icp998-1012.html
   My bibliography  Save this article

Missing value imputation for short to mid-term horizontal solar irradiance data

Author

Listed:
  • Demirhan, Haydar
  • Renwick, Zoe

Abstract

Improving the accuracy of solar irradiance forecasting has become crucial since the use of solar energy power has become more accessible due to increased efficiency and decreased costs associated with its production. Data quality and availability are essential to producing accurate solar irradiance forecasts. In this article, we focus on the estimation of missing values in minutely, hourly, daily, and weekly solar irradiance series using an extensive number of imputation methods. We compare the accuracy of 36 imputation methods for solar irradiance series over a real dataset recorded in Australia under 16 experimental conditions. The experiments are run in a semi-Monte Carlo setting, in which missing values are randomly generated in the solar irradiance series. Our results identify the most reliable and robust approaches for the imputation of solar irradiance for each of the mentioned frequencies. While linear and Stineman interpolations and Kalman filtering with structural model and smoothing are found accurate for minutely and hourly series, weighted moving average gives the highly precise imputations for daily and weekly solar irradiance.

Suggested Citation

  • Demirhan, Haydar & Renwick, Zoe, 2018. "Missing value imputation for short to mid-term horizontal solar irradiance data," Applied Energy, Elsevier, vol. 225(C), pages 998-1012.
    • Handle: RePEc:eee:appene:v:225:y:2018:i:c:p:998-1012
    DOI: 10.1016/j.apenergy.2018.05.054
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261918307645
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2018.05.054?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Baser, Furkan & Demirhan, Haydar, 2017. "A fuzzy regression with support vector machine approach to the estimation of horizontal global solar radiation," Energy, Elsevier, vol. 123(C), pages 229-240.
    2. Halabi, Laith M. & Mekhilef, Saad & Hossain, Monowar, 2018. "Performance evaluation of hybrid adaptive neuro-fuzzy inference system models for predicting monthly global solar radiation," Applied Energy, Elsevier, vol. 213(C), pages 247-261.
    3. Pashiardis, S. & Kalogirou, S.A. & Pelengaris, A., 2017. "Statistical analysis for the characterization of solar energy utilization and inter-comparison of solar radiation at two sites in Cyprus," Applied Energy, Elsevier, vol. 190(C), pages 1138-1158.
    4. Anthony Mouraud, 2017. "Innovative time series forecasting: auto regressive moving average vs deep networks," Entrepreneurship and Sustainability Issues, VsI Entrepreneurship and Sustainability Center, vol. 4(3), pages 282-293, March.
    5. Salcedo-Sanz, Sancho & Deo, Ravinesh C. & Cornejo-Bueno, Laura & Camacho-Gómez, Carlos & Ghimire, Sujan, 2018. "An efficient neuro-evolutionary hybrid modelling mechanism for the estimation of daily global solar radiation in the Sunshine State of Australia," Applied Energy, Elsevier, vol. 209(C), pages 79-94.
    6. Patrick, Joshua D. & Harvill, Jane L. & Hansen, Clifford W., 2016. "A semiparametric spatio-temporal model for solar irradiance data," Renewable Energy, Elsevier, vol. 87(P1), pages 15-30.
    7. Hassan, Muhammed A. & Khalil, A. & Kaseb, S. & Kassem, M.A., 2017. "Exploring the potential of tree-based ensemble methods in solar radiation modeling," Applied Energy, Elsevier, vol. 203(C), pages 897-916.
    8. Alessandrini, S. & Delle Monache, L. & Sperati, S. & Cervone, G., 2015. "An analog ensemble for short-term probabilistic solar power forecast," Applied Energy, Elsevier, vol. 157(C), pages 95-110.
    9. Trapero, Juan R. & Kourentzes, Nikolaos & Martin, A., 2015. "Short-term solar irradiation forecasting based on Dynamic Harmonic Regression," Energy, Elsevier, vol. 84(C), pages 289-295.
    10. Deo, Ravinesh C. & Wen, Xiaohu & Qi, Feng, 2016. "A wavelet-coupled support vector machine model for forecasting global incident solar radiation using limited meteorological dataset," Applied Energy, Elsevier, vol. 168(C), pages 568-593.
    11. Chao Chen & Jamie Twycross & Jonathan M Garibaldi, 2017. "A new accuracy measure based on bounded relative error for time series forecasting," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-23, March.
    12. Akçay, Hüseyin & Filik, Tansu, 2017. "Short-term wind speed forecasting by spectral analysis from long-term observations with missing values," Applied Energy, Elsevier, vol. 191(C), pages 653-662.
    13. Hassan, Gasser E. & Youssef, M. Elsayed & Mohamed, Zahraa E. & Ali, Mohamed A. & Hanafy, Ahmed A., 2016. "New Temperature-based Models for Predicting Global Solar Radiation," Applied Energy, Elsevier, vol. 179(C), pages 437-450.
    14. Barbieri, Florian & Rajakaruna, Sumedha & Ghosh, Arindam, 2017. "Very short-term photovoltaic power forecasting with cloud modeling: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 75(C), pages 242-263.
    15. Boland, John, 2015. "Spatial-temporal forecasting of solar radiation," Renewable Energy, Elsevier, vol. 75(C), pages 607-616.
    16. Voyant, Cyril & Soubdhan, Ted & Lauret, Philippe & David, Mathieu & Muselli, Marc, 2015. "Statistical parameters as a means to a priori assess the accuracy of solar forecasting models," Energy, Elsevier, vol. 90(P1), pages 671-679.
    17. Diagne, Maimouna & David, Mathieu & Lauret, Philippe & Boland, John & Schmutz, Nicolas, 2013. "Review of solar irradiance forecasting methods and a proposition for small-scale insular grids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 65-76.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yeom, Jong-Min & Deo, Ravinesh C. & Adamwoski, Jan F. & Chae, Taebyeong & Kim, Dong-Su & Han, Kyung-Soo & Kim, Do-Yong, 2020. "Exploring solar and wind energy resources in North Korea with COMS MI geostationary satellite data coupled with numerical weather prediction reanalysis variables," Renewable and Sustainable Energy Reviews, Elsevier, vol. 119(C).
    2. Jeong, Dongyeon & Park, Chiwoo & Ko, Young Myoung, 2021. "Missing data imputation using mixture factor analysis for building electric load data," Applied Energy, Elsevier, vol. 304(C).
    3. Zang, Haixiang & Cheng, Lilin & Ding, Tao & Cheung, Kwok W. & Wang, Miaomiao & Wei, Zhinong & Sun, Guoqiang, 2019. "Estimation and validation of daily global solar radiation by day of the year-based models for different climates in China," Renewable Energy, Elsevier, vol. 135(C), pages 984-1003.
    4. Endah Kristiani & Hao Lin & Jwu-Rong Lin & Yen-Hsun Chuang & Chin-Yin Huang & Chao-Tung Yang, 2022. "Short-Term Prediction of PM 2.5 Using LSTM Deep Learning Methods," Sustainability, MDPI, vol. 14(4), pages 1-29, February.
    5. Robert Basmadjian & Amirhossein Shaafieyoun & Sahib Julka, 2021. "Day-Ahead Forecasting of the Percentage of Renewables Based on Time-Series Statistical Methods," Energies, MDPI, vol. 14(21), pages 1-23, November.
    6. Alhamwi, Alaa & Medjroubi, Wided & Vogt, Thomas & Agert, Carsten, 2018. "Modelling urban energy requirements using open source data and models," Applied Energy, Elsevier, vol. 231(C), pages 1100-1108.
    7. Sascha Lindig & Atse Louwen & David Moser & Marko Topic, 2020. "Outdoor PV System Monitoring—Input Data Quality, Data Imputation and Filtering Approaches," Energies, MDPI, vol. 13(19), pages 1-18, September.
    8. Pedregal, Diego J. & Trapero, Juan R., 2021. "Adjusted combination of moving averages: A forecasting system for medium-term solar irradiance," Applied Energy, Elsevier, vol. 298(C).
    9. Hoyos-Gómez, Laura S. & Ruiz-Muñoz, Jose F. & Ruiz-Mendoza, Belizza J., 2022. "Short-term forecasting of global solar irradiance in tropical environments with incomplete data," Applied Energy, Elsevier, vol. 307(C).
    10. Lei Fu & Yiling Yang & Xiaolong Yao & Xufen Jiao & Tiantian Zhu, 2019. "A Regional Photovoltaic Output Prediction Method Based on Hierarchical Clustering and the mRMR Criterion," Energies, MDPI, vol. 12(20), pages 1-23, October.
    11. Chuanjie Xie & Chong Huang & Deqiang Zhang & Wei He, 2021. "BiLSTM-I: A Deep Learning-Based Long Interval Gap-Filling Method for Meteorological Observation Data," IJERPH, MDPI, vol. 18(19), pages 1-12, September.
    12. Huyghues-Beaufond, Nathalie & Tindemans, Simon & Falugi, Paola & Sun, Mingyang & Strbac, Goran, 2020. "Robust and automatic data cleansing method for short-term load forecasting of distribution feeders," Applied Energy, Elsevier, vol. 261(C).
    13. Yagli, Gokhan Mert & Yang, Dazhi & Gandhi, Oktoviano & Srinivasan, Dipti, 2020. "Can we justify producing univariate machine-learning forecasts with satellite-derived solar irradiance?," Applied Energy, Elsevier, vol. 259(C).
    14. Kasım Zor & Özgür Çelik & Oğuzhan Timur & Ahmet Teke, 2020. "Short-Term Building Electrical Energy Consumption Forecasting by Employing Gene Expression Programming and GMDH Networks," Energies, MDPI, vol. 13(5), pages 1-24, March.
    15. Zhao, He & Huang, Xiaoqiao & Xiao, Zenan & Shi, Haoyuan & Li, Chengli & Tai, Yonghang, 2024. "Week-ahead hourly solar irradiation forecasting method based on ICEEMDAN and TimesNet networks," Renewable Energy, Elsevier, vol. 220(C).
    16. Agnieszka Kowalska-Styczeń & Tomasz Owczarek & Janusz Siwy & Adam Sojda & Maciej Wolny, 2022. "Analysis of Business Customers’ Energy Consumption Data Registered by Trading Companies in Poland," Energies, MDPI, vol. 15(14), pages 1-23, July.
    17. Houtian Tang & Yuanlai Wu & Jinxiu Chen & Liuxin Deng & Minjie Zeng, 2022. "How Does Change in Rural Residential Land Affect Cultivated Land Use Efficiency? An Empirical Study Based on 42 Cities in the Middle Reaches of the Yangtze River," Land, MDPI, vol. 11(12), pages 1-20, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Qian & Wu, Zhou & Xia, Xiaohua, 2018. "Estimate and characterize PV power at demand-side hybrid system," Applied Energy, Elsevier, vol. 218(C), pages 66-77.
    2. Michel Fliess & Cédric Join & Cyril Voyant, 2018. "Prediction bands for solar energy: New short-term time series forecasting techniques," Post-Print hal-01736518, HAL.
    3. Ping-Huan Kuo & Chiou-Jye Huang, 2018. "A Green Energy Application in Energy Management Systems by an Artificial Intelligence-Based Solar Radiation Forecasting Model," Energies, MDPI, vol. 11(4), pages 1-15, April.
    4. Elsinga, Boudewijn & van Sark, Wilfried G.J.H.M., 2017. "Short-term peer-to-peer solar forecasting in a network of photovoltaic systems," Applied Energy, Elsevier, vol. 206(C), pages 1464-1483.
    5. Àlex Alonso & Jordi de la Hoz & Helena Martín & Sergio Coronas & Pep Salas & José Matas, 2020. "A Comprehensive Model for the Design of a Microgrid under Regulatory Constraints Using Synthetical Data Generation and Stochastic Optimization," Energies, MDPI, vol. 13(21), pages 1-26, October.
    6. Feng, Yu & Hao, Weiping & Li, Haoru & Cui, Ningbo & Gong, Daozhi & Gao, Lili, 2020. "Machine learning models to quantify and map daily global solar radiation and photovoltaic power," Renewable and Sustainable Energy Reviews, Elsevier, vol. 118(C).
    7. Olubayo M. Babatunde & Josiah L. Munda & Yskandar Hamam, 2020. "Exploring the Potentials of Artificial Neural Network Trained with Differential Evolution for Estimating Global Solar Radiation," Energies, MDPI, vol. 13(10), pages 1-18, May.
    8. Elham Alzain & Shaha Al-Otaibi & Theyazn H. H. Aldhyani & Ali Saleh Alshebami & Mohammed Amin Almaiah & Mukti E. Jadhav, 2023. "Revolutionizing Solar Power Production with Artificial Intelligence: A Sustainable Predictive Model," Sustainability, MDPI, vol. 15(10), pages 1-21, May.
    9. Anh Ngoc-Lan Huynh & Ravinesh C. Deo & Duc-Anh An-Vo & Mumtaz Ali & Nawin Raj & Shahab Abdulla, 2020. "Near Real-Time Global Solar Radiation Forecasting at Multiple Time-Step Horizons Using the Long Short-Term Memory Network," Energies, MDPI, vol. 13(14), pages 1-30, July.
    10. Si-Ya Wang & Jun Qiu & Fang-Fang Li, 2018. "Hybrid Decomposition-Reconfiguration Models for Long-Term Solar Radiation Prediction Only Using Historical Radiation Records," Energies, MDPI, vol. 11(6), pages 1-17, May.
    11. Khan, Zulfiqar Ahmad & Hussain, Tanveer & Baik, Sung Wook, 2023. "Dual stream network with attention mechanism for photovoltaic power forecasting," Applied Energy, Elsevier, vol. 338(C).
    12. Qiu, Rangjian & Li, Longan & Wu, Lifeng & Agathokleous, Evgenios & Liu, Chunwei & Zhang, Baozhong & Luo, Yufeng & Sun, Shanlei, 2022. "Modeling daily global solar radiation using only temperature data: Past, development, and future," Renewable and Sustainable Energy Reviews, Elsevier, vol. 163(C).
    13. Li, Shuai & Ma, Hongjie & Li, Weiyi, 2017. "Typical solar radiation year construction using k-means clustering and discrete-time Markov chain," Applied Energy, Elsevier, vol. 205(C), pages 720-731.
    14. Martins, Guilherme Santos & Giesbrecht, Mateus, 2023. "Hybrid approaches based on Singular Spectrum Analysis and k- Nearest Neighbors for clearness index forecasting," Renewable Energy, Elsevier, vol. 219(P1).
    15. Zang, Haixiang & Cheng, Lilin & Ding, Tao & Cheung, Kwok W. & Wang, Miaomiao & Wei, Zhinong & Sun, Guoqiang, 2019. "Estimation and validation of daily global solar radiation by day of the year-based models for different climates in China," Renewable Energy, Elsevier, vol. 135(C), pages 984-1003.
    16. van der Meer, D.W. & Widén, J. & Munkhammar, J., 2018. "Review on probabilistic forecasting of photovoltaic power production and electricity consumption," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1484-1512.
    17. Zang, Haixiang & Cheng, Lilin & Ding, Tao & Cheung, Kwok W. & Wang, Miaomiao & Wei, Zhinong & Sun, Guoqiang, 2020. "Application of functional deep belief network for estimating daily global solar radiation: A case study in China," Energy, Elsevier, vol. 191(C).
    18. Theo, Wai Lip & Lim, Jeng Shiun & Ho, Wai Shin & Hashim, Haslenda & Lee, Chew Tin, 2017. "Review of distributed generation (DG) system planning and optimisation techniques: Comparison of numerical and mathematical modelling methods," Renewable and Sustainable Energy Reviews, Elsevier, vol. 67(C), pages 531-573.
    19. Alizamir, Meysam & Kim, Sungwon & Kisi, Ozgur & Zounemat-Kermani, Mohammad, 2020. "A comparative study of several machine learning based non-linear regression methods in estimating solar radiation: Case studies of the USA and Turkey regions," Energy, Elsevier, vol. 197(C).
    20. Bouchouicha, Kada & Hassan, Muhammed A. & Bailek, Nadjem & Aoun, Nouar, 2019. "Estimating the global solar irradiation and optimizing the error estimates under Algerian desert climate," Renewable Energy, Elsevier, vol. 139(C), pages 844-858.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:225:y:2018:i:c:p:998-1012. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.