IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v236y2019icp196-210.html
   My bibliography  Save this article

On the impact of outlier filtering on the electricity price forecasting accuracy

Author

Listed:
  • Afanasyev, Dmitriy O.
  • Fedorova, Elena A.

Abstract

Increasing the accuracy of short-term electricity price forecasting allows day-ahead power market participants to obtain a positive economic effect by bidding close to the equilibrium price. However the electricity price time-series is generally infested with extreme values due to high price volatility. This paper discusses the impact of outlier filtering on forecasting accuracy based on a recently introduced seasonal component autoregressive model. We consider such methods of outlier detection (with a priori defined cut-off parameter) as threshold, standard deviation, percentage, recursive, and moving filter on prices. It is shown that such data pre-processing often leads to the forecasting accuracy gain while the error decrease (relative to the approach without filtering) in a number of cases may reach 1.8–1.9% of the average weekly price (in absolute values). For an a priori defined cut-off parameter, the simple threshold and standard deviation filter on prices outperform other considered methods, and yield to the accuracy gain in 63% and 67% of cases, correspondingly. At the same time, in case of the out-of-sample filter parameter grid-optimization all of the methods demonstrate comparable prediction power (equal to the marginal performance). But, practically speaking, such optimization is time-consuming and cannot be carried out on unavailable future data. As an competitive alternative, we propose a combined filter on prices based on a committee machine which uses the results of individual non-optimized algorithms and is not time-consuming, but gives accuracy comparable to the best one obtained for each of the studied electricity markets and leads to forecast gain in 63% of the considered cases.

Suggested Citation

  • Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
  • Handle: RePEc:eee:appene:v:236:y:2019:i:c:p:196-210
    DOI: 10.1016/j.apenergy.2018.11.076
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261918317926
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2018.11.076?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Misiorek Adam & Trueck Stefan & Weron Rafal, 2006. "Point and Interval Forecasting of Spot Electricity Prices: Linear vs. Non-Linear Time Series Models," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 10(3), pages 1-36, September.
    2. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    3. Garcia, Márcio G.P. & Medeiros, Marcelo C. & Vasconcelos, Gabriel F.R., 2017. "Real-time inflation forecasting with high-dimensional models: The case of Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 679-693.
    4. Alvaro Cartea & Marcelo Figueroa, 2005. "Pricing in Electricity Markets: A Mean Reverting Jump Diffusion Model with Seasonality," Applied Mathematical Finance, Taylor & Francis Journals, vol. 12(4), pages 313-335.
    5. Peter Reinhard Hansen & Asger Lunde & James M. Nason, 2005. "Model confidence sets for forecasting models," FRB Atlanta Working Paper 2005-07, Federal Reserve Bank of Atlanta.
    6. Nowotarski, Jakub & Raviv, Eran & Trück, Stefan & Weron, Rafał, 2014. "An empirical comparison of alternative schemes for combining electricity spot price forecasts," Energy Economics, Elsevier, vol. 46(C), pages 395-412.
    7. Sergey Voronin & Jarmo Partanen, 2013. "Price Forecasting in the Day-Ahead Energy Market by an Iterative Method with Separate Normal Price and Price Spike Frameworks," Energies, MDPI, vol. 6(11), pages 1-24, November.
    8. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    9. Tan, Zhongfu & Zhang, Jinliang & Wang, Jianhui & Xu, Jun, 2010. "Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models," Applied Energy, Elsevier, vol. 87(11), pages 3606-3610, November.
    10. Nowotarski, Jakub & Tomczyk, Jakub & Weron, Rafał, 2013. "Robust estimation and forecasting of the long-term seasonal component of electricity spot prices," Energy Economics, Elsevier, vol. 39(C), pages 13-27.
    11. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    12. He, Kaijian & Yu, Lean & Tang, Ling, 2015. "Electricity price forecasting with a BED (Bivariate EMD Denoising) methodology," Energy, Elsevier, vol. 91(C), pages 601-609.
    13. Karakatsani, Nektaria V. & Bunn, Derek W., 2008. "Forecasting electricity prices: The impact of fundamentals and time-varying coefficients," International Journal of Forecasting, Elsevier, vol. 24(4), pages 764-785.
    14. Bordignon, Silvano & Bunn, Derek W. & Lisi, Francesco & Nan, Fany, 2013. "Combining day-ahead forecasts for British electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 88-103.
    15. Weron, Rafal, 2008. "Market price of risk implied by Asian-style electricity options and futures," Energy Economics, Elsevier, vol. 30(3), pages 1098-1115, May.
    16. Keles, Dogan & Scelle, Jonathan & Paraschiv, Florentina & Fichtner, Wolf, 2016. "Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks," Applied Energy, Elsevier, vol. 162(C), pages 218-230.
    17. Trueck, Stefan & Weron, Rafal & Wolff, Rodney, 2007. "Outlier Treatment and Robust Approaches for Modeling Electricity Spot Prices," MPRA Paper 4711, University Library of Munich, Germany.
    18. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    19. Weron, Rafal & Misiorek, Adam, 2008. "Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models," International Journal of Forecasting, Elsevier, vol. 24(4), pages 744-763.
    20. Samuels, Jon D. & Sekkel, Rodrigo M., 2017. "Model Confidence Sets and forecast combination," International Journal of Forecasting, Elsevier, vol. 33(1), pages 48-60.
    21. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    22. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    23. Janczura, Joanna & Weron, Rafal, 2010. "An empirical comparison of alternate regime-switching models for electricity spot prices," Energy Economics, Elsevier, vol. 32(5), pages 1059-1073, September.
    24. Crespo Cuaresma, Jesús & Hlouskova, Jaroslava & Kossmeier, Stephan & Obersteiner, Michael, 2004. "Forecasting electricity spot-prices using linear univariate time-series models," Applied Energy, Elsevier, vol. 77(1), pages 87-106, January.
    25. Lisi, Francesco & Nan, Fany, 2014. "Component estimation for electricity prices: Procedures and comparisons," Energy Economics, Elsevier, vol. 44(C), pages 143-159.
    26. Bierbrauer, Michael & Menn, Christian & Rachev, Svetlozar T. & Truck, Stefan, 2007. "Spot and derivative pricing in the EEX power market," Journal of Banking & Finance, Elsevier, vol. 31(11), pages 3462-3485, November.
    27. De Jong Cyriel, 2006. "The Nature of Power Spikes: A Regime-Switch Approach," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 10(3), pages 1-28, September.
    28. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    29. Xiao, Liye & Shao, Wei & Yu, Mengxia & Ma, Jing & Jin, Congjun, 2017. "Research and application of a hybrid wavelet neural network model with the improved cuckoo search algorithm for electrical power system forecasting," Applied Energy, Elsevier, vol. 198(C), pages 203-222.
    30. Adam Misiorek & Rafal Weron, 2006. "Interval forecasting of spot electricity prices," HSC Research Reports HSC/06/05, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    31. Fanone, Enzo & Gamba, Andrea & Prokopczuk, Marcel, 2013. "The case of negative day-ahead electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 22-34.
    32. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2016. "The long-term trends on the electricity markets: Comparison of empirical mode and wavelet decompositions," Energy Economics, Elsevier, vol. 56(C), pages 432-442.
    33. Rafal Weron, 2006. "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach," HSC Books, Hugo Steinhaus Center, Wroclaw University of Science and Technology, number hsbook0601, December.
    34. Yang, Zhang & Ce, Li & Lian, Li, 2017. "Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods," Applied Energy, Elsevier, vol. 190(C), pages 291-305.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gang Zhou & Zhongjie Han & Jin Fu & Guan Hua Xu & Chengjin Ye, 2020. "Iterative Online Fault Identification Scheme for High-Voltage Circuit Breaker Utilizing a Lost Data Repair Technique," Energies, MDPI, vol. 13(13), pages 1-15, June.
    2. Angelica Gianfreda & Francesco Ravazzolo & Luca Rossini, 2023. "Large Time‐Varying Volatility Models for Hourly Electricity Prices," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 85(3), pages 545-573, June.
    3. Li, Dongxin & Zhang, Li & Li, Lihong, 2023. "Forecasting stock volatility with economic policy uncertainty: A smooth transition GARCH-MIDAS model," International Review of Financial Analysis, Elsevier, vol. 88(C).
    4. Billé, Anna Gloria & Gianfreda, Angelica & Del Grosso, Filippo & Ravazzolo, Francesco, 2023. "Forecasting electricity prices with expert, linear, and nonlinear models," International Journal of Forecasting, Elsevier, vol. 39(2), pages 570-586.
    5. Daniel Manfre Jaimes & Manuel Zamudio López & Hamidreza Zareipour & Mike Quashie, 2023. "A Hybrid Model for Multi-Day-Ahead Electricity Price Forecasting considering Price Spikes," Forecasting, MDPI, vol. 5(3), pages 1-23, July.
    6. Bartosz Uniejewski, 2023. "Smoothing Quantile Regression Averaging: A new approach to probabilistic forecasting of electricity prices," Papers 2302.00411, arXiv.org, revised Nov 2024.
    7. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    8. Uniejewski, Bartosz & Weron, Rafał, 2021. "Regularized quantile regression averaging for probabilistic electricity price forecasting," Energy Economics, Elsevier, vol. 95(C).
    9. Beltrán, Sergio & Castro, Alain & Irizar, Ion & Naveran, Gorka & Yeregui, Imanol, 2022. "Framework for collaborative intelligence in forecasting day-ahead electricity price," Applied Energy, Elsevier, vol. 306(PA).
    10. Rainer Baule & Michael Naumann, 2021. "Volatility and Dispersion of Hourly Electricity Contracts on the German Continuous Intraday Market," Energies, MDPI, vol. 14(22), pages 1-24, November.
    11. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    12. AL-Alimi, Dalal & AlRassas, Ayman Mutahar & Al-qaness, Mohammed A.A. & Cai, Zhihua & Aseeri, Ahmad O. & Abd Elaziz, Mohamed & Ewees, Ahmed A., 2023. "TLIA: Time-series forecasting model using long short-term memory integrated with artificial neural networks for volatile energy markets," Applied Energy, Elsevier, vol. 343(C).
    13. Shao, Zhen & Yang, Yudie & Zheng, Qingru & Zhou, Kaile & Liu, Chen & Yang, Shanlin, 2022. "A pattern classification methodology for interval forecasts of short-term electricity prices based on hybrid deep neural networks: A comparative analysis," Applied Energy, Elsevier, vol. 327(C).
    14. Sheybanivaziri, Samaneh & Le Dréau, Jérôme & Kazmi, Hussain, 2024. "Forecasting price spikes in day-ahead electricity markets: techniques, challenges, and the road ahead," Discussion Papers 2024/1, Norwegian School of Economics, Department of Business and Management Science.
    15. Elmore, Clay T. & Dowling, Alexander W., 2021. "Learning spatiotemporal dynamics in wholesale energy markets with dynamic mode decomposition," Energy, Elsevier, vol. 232(C).
    16. Qiao, Weibiao & Yang, Zhe, 2020. "Forecast the electricity price of U.S. using a wavelet transform-based hybrid model," Energy, Elsevier, vol. 193(C).
    17. Arkadiusz Jędrzejewski & Grzegorz Marcjasz & Rafał Weron, 2021. "Importance of the Long-Term Seasonal Component in Day-Ahead Electricity Price Forecasting Revisited: Parameter-Rich Models Estimated via the LASSO," Energies, MDPI, vol. 14(11), pages 1-17, June.
    18. Manuel Zamudio López & Hamidreza Zareipour & Mike Quashie, 2024. "Forecasting the Occurrence of Electricity Price Spikes: A Statistical-Economic Investigation Study," Forecasting, MDPI, vol. 6(1), pages 1-23, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    2. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    3. Ziel, Florian & Weron, Rafał, 2018. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks," Energy Economics, Elsevier, vol. 70(C), pages 396-420.
    4. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    5. Florian Ziel & Rafal Weron, 2016. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate models," HSC Research Reports HSC/16/08, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    6. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2020. "Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts?," International Journal of Forecasting, Elsevier, vol. 36(2), pages 466-479.
    7. Grzegorz Marcjasz & Bartosz Uniejewski & Rafal Weron, 2017. "Importance of the long-term seasonal component in day-ahead electricity price forecasting revisited: Neural network models," HSC Research Reports HSC/17/03, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
    8. Grzegorz Marcjasz & Tomasz Serafin & Rafał Weron, 2018. "Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 11(9), pages 1-20, September.
    9. Alexandre Lucas & Konstantinos Pegios & Evangelos Kotsakis & Dan Clarke, 2020. "Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression," Energies, MDPI, vol. 13(20), pages 1-16, October.
    10. Avci, Ezgi & Ketter, Wolfgang & van Heck, Eric, 2018. "Managing electricity price modeling risk via ensemble forecasting: The case of Turkey," Energy Policy, Elsevier, vol. 123(C), pages 390-403.
    11. Uniejewski, Bartosz & Marcjasz, Grzegorz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting: Part II — Probabilistic forecasting," Energy Economics, Elsevier, vol. 79(C), pages 171-182.
    12. Luigi Grossi & Fany Nan, 2018. "The influence of renewables on electricity price forecasting: a robust approach," Working Papers 2018/10, Institut d'Economia de Barcelona (IEB).
    13. Grossi, Luigi & Nan, Fany, 2019. "Robust forecasting of electricity prices: Simulations, models and the impact of renewable sources," Technological Forecasting and Social Change, Elsevier, vol. 141(C), pages 305-318.
    14. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    15. Weron, Rafał & Zator, Michał, 2015. "A note on using the Hodrick–Prescott filter in electricity markets," Energy Economics, Elsevier, vol. 48(C), pages 1-6.
    16. Luigi Grossi & Fany Nan, 2017. "Forecasting electricity prices through robust nonlinear models," Working Papers 06/2017, University of Verona, Department of Economics.
    17. Bartosz Uniejewski & Jakub Nowotarski & Rafał Weron, 2016. "Automated Variable Selection and Shrinkage for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 9(8), pages 1-22, August.
    18. Bartosz Uniejewski & Rafał Weron, 2018. "Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models," Energies, MDPI, vol. 11(8), pages 1-26, August.
    19. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    20. Lisi, Francesco & Nan, Fany, 2014. "Component estimation for electricity prices: Procedures and comparisons," Energy Economics, Elsevier, vol. 44(C), pages 143-159.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:236:y:2019:i:c:p:196-210. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.