IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-03359179.html
   My bibliography  Save this paper

Bewertung der Verzerrung von Punktprognosen über mehrere Zeitreihen hinweg: Maßnahmen und visuelle Werkzeuge
[Assessing point forecast bias across multiple time series: Measures and visual tools]

Author

Listed:
  • Andrey Davydenko

    (Independent Researcher)

  • Paul Goodwin

    (University of Bath [Bath])

Abstract

Measuring bias is important as it helps identify flaws in quantitative forecasting methods or judgmental forecasts. It can, therefore, potentially help improve forecasts. Despite this, bias tends to be under-represented in the literature: many studies focus solely on measuring accuracy. Methods for assessing bias in single series are relatively well-known and well-researched, but for datasets containing thousands of observations for multiple series, the methodology for measuring and reporting bias is less obvious. We compare alternative approaches against a number of criteria when rolling-origin point forecasts are available for different forecasting methods and for multiple horizons over multiple series. We focus on relatively simple, yet interpretable and easy-to-implement metrics and visualization tools that are likely to be applicable in practice. To study the statistical properties of alternative measures we use theoretical concepts and simulation experiments based on artificial data with predetermined features. We describe the difference between mean and median bias, describe the connection between metrics for accuracy and bias, provide suitable bias measures depending on the loss function used to optimise forecasts, and suggest which measures for accuracy should be used to accompany bias indicators. We propose several new measures and provide our recommendations on how to evaluate forecast bias across multiple series.

Suggested Citation

  • Andrey Davydenko & Paul Goodwin, 2021. "Bewertung der Verzerrung von Punktprognosen über mehrere Zeitreihen hinweg: Maßnahmen und visuelle Werkzeuge [Assessing point forecast bias across multiple time series: Measures and visual tools]," Post-Print hal-03359179, HAL.
  • Handle: RePEc:hal:journl:hal-03359179
    DOI: 10.5539/ijsp.v10n5p46
    Note: View the original document on HAL open archive server: https://hal.science/hal-03359179
    as

    Download full text from publisher

    File URL: https://hal.science/hal-03359179/document
    Download Restriction: no

    File URL: https://libkey.io/10.5539/ijsp.v10n5p46?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    2. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    3. Unknown, 2005. "Forward," 2005 Conference: Slovenia in the EU - Challenges for Agriculture, Food Science and Rural Affairs, November 10-11, 2005, Moravske Toplice, Slovenia 183804, Slovenian Association of Agricultural Economists (DAES).
    4. Goodwin, Paul, 2000. "Correct or combine? Mechanically integrating judgmental forecasts with statistical methods," International Journal of Forecasting, Elsevier, vol. 16(2), pages 261-275.
    5. Davydenko, Andrey & Goodwin, Paul, 2021. "Assessing Point Forecast Bias Across Multiple Time Series: Measures and Visual Tools," SocArXiv jhtzw, Center for Open Science.
    6. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2018. "The M4 Competition: Results, findings, conclusion and way forward," International Journal of Forecasting, Elsevier, vol. 34(4), pages 802-808.
    7. Fildes, Robert & Goodwin, Paul, 2021. "Stability in the inefficient use of forecasting systems: A case study in a supply chain company," International Journal of Forecasting, Elsevier, vol. 37(2), pages 1031-1046.
    8. Andrey Davydenko & Paul Goodwin, 2021. "Assessing Point Forecast Bias Across Multiple Time Series: Measures and Visual Tools," International Journal of Statistics and Probability, Canadian Center of Science and Education, vol. 10(5), pages 1-46, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrey Davydenko & Paul Goodwin, 2021. "Assessing Point Forecast Bias Across Multiple Time Series: Measures and Visual Tools," International Journal of Statistics and Probability, Canadian Center of Science and Education, vol. 10(5), pages 1-46, September.
    2. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    3. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2022. "Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1283-1318.
    4. Alvarado-Valencia, Jorge & Barrero, Lope H. & Önkal, Dilek & Dennerlein, Jack T., 2017. "Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting," International Journal of Forecasting, Elsevier, vol. 33(1), pages 298-313.
    5. Trapero, Juan R. & Pedregal, Diego J. & Fildes, R. & Kourentzes, N., 2013. "Analysis of judgmental adjustments in the presence of promotions," International Journal of Forecasting, Elsevier, vol. 29(2), pages 234-243.
    6. Hewamalage, Hansika & Bergmeir, Christoph & Bandara, Kasun, 2021. "Recurrent Neural Networks for Time Series Forecasting: Current status and future directions," International Journal of Forecasting, Elsevier, vol. 37(1), pages 388-427.
    7. Philippe St-Aubin & Bruno Agard, 2022. "Precision and Reliability of Forecasts Performance Metrics," Forecasting, MDPI, vol. 4(4), pages 1-22, October.
    8. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    9. Syntetos, Aris A. & Kholidasari, Inna & Naim, Mohamed M., 2016. "The effects of integrating management judgement into OUT levels: In or out of context?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 853-863.
    10. Thiyanga S. Talagala & Feng Li & Yanfei Kang, 2019. "Feature-based Forecast-Model Performance Prediction," Monash Econometrics and Business Statistics Working Papers 21/19, Monash University, Department of Econometrics and Business Statistics.
    11. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2022. "Forecasting natural gas consumption using Bagging and modified regularization techniques," Energy Economics, Elsevier, vol. 106(C).
    12. Huber, Jakob & Stuckenschmidt, Heiner, 2020. "Daily retail demand forecasting using machine learning with emphasis on calendric special days," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1420-1438.
    13. Spiliotis, Evangelos & Petropoulos, Fotios, 2024. "On the update frequency of univariate forecasting models," European Journal of Operational Research, Elsevier, vol. 314(1), pages 111-121.
    14. Voyant, Cyril & Notton, Gilles & Duchaud, Jean-Laurent & Gutiérrez, Luis Antonio García & Bright, Jamie M. & Yang, Dazhi, 2022. "Benchmarks for solar radiation time series forecasting," Renewable Energy, Elsevier, vol. 191(C), pages 747-762.
    15. De Baets, Shari & Harvey, Nigel, 2018. "Forecasting from time series subject to sporadic perturbations: Effectiveness of different types of forecasting support," International Journal of Forecasting, Elsevier, vol. 34(2), pages 163-180.
    16. Niematallah Elamin & Mototsugu Fukushige, 2017. "Integrating judgment in statistical demand forecasting: An approach to confront uncertainty," Discussion Papers in Economics and Business 17-20, Osaka University, Graduate School of Economics.
    17. Evangelos Spiliotis & Fotios Petropoulos & Vassilios Assimakopoulos, 2023. "On the Disagreement of Forecasting Model Selection Criteria," Forecasting, MDPI, vol. 5(2), pages 1-12, June.
    18. R Fildes & K Nikolopoulos & S F Crone & A A Syntetos, 2008. "Forecasting and operational research: a review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(9), pages 1150-1172, September.
    19. Meira, Erick & Cyrino Oliveira, Fernando Luiz & Jeon, Jooyoung, 2021. "Treating and Pruning: New approaches to forecasting model selection and combination using prediction intervals," International Journal of Forecasting, Elsevier, vol. 37(2), pages 547-568.
    20. Karamaziotis, Panagiotis I. & Raptis, Achilleas & Nikolopoulos, Konstantinos & Litsiou, Konstantia & Assimakopoulos, Vassilis, 2020. "An empirical investigation of water consumption forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(2), pages 588-606.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-03359179. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.