The M5 uncertainty competition: Results, findings and conclusions
Author
Abstract
Suggested Citation
DOI: 10.1016/j.ijforecast.2021.10.009
Download full text from publisher
As the access to this document is restricted, you may want to search for a different version of it.
References listed on IDEAS
- Anil Gaba & Dana G. Popescu & Zhi Chen, 2019. "Assessing Uncertainty from Point Forecasts," Management Science, INFORMS, vol. 65(1), pages 90-106, January.
- Smyl, Slawek, 2020. "A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting," International Journal of Forecasting, Elsevier, vol. 36(1), pages 75-85.
- Makridakis, Spyros & Hibon, Michele & Lusk, Ed & Belhadjali, Moncef, 1987. "Confidence intervals: An empirical investigation of the series in the M-competition," International Journal of Forecasting, Elsevier, vol. 3(3-4), pages 489-508.
- Hyndman, Rob J. & Koehler, Anne B. & Snyder, Ralph D. & Grose, Simone, 2002.
"A state space framework for automatic forecasting using exponential smoothing methods,"
International Journal of Forecasting, Elsevier, vol. 18(3), pages 439-454.
- Hyndman, R.J. & Koehler, A.B. & Snyder, R.D. & Grose, S., 2000. "A State Space Framework for Automatic Forecasting Using Exponential Smoothing Methods," Monash Econometrics and Business Statistics Working Papers 9/00, Monash University, Department of Econometrics and Business Statistics.
- Hong, Tao & Xie, Jingrui & Black, Jonathan, 2019. "Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1389-1399.
- Montero-Manso, Pablo & Athanasopoulos, George & Hyndman, Rob J. & Talagala, Thiyanga S., 2020.
"FFORMA: Feature-based forecast model averaging,"
International Journal of Forecasting, Elsevier, vol. 36(1), pages 86-92.
- Pablo Montero-Manso & George Athanasopoulos & Rob J Hyndman & Thiyanga S Talagala, 2018. "FFORMA: Feature-based forecast model averaging," Monash Econometrics and Business Statistics Working Papers 19/18, Monash University, Department of Econometrics and Business Statistics.
- Ivan Svetunkov & Fotios Petropoulos, 2018. "Old dog, new tricks: a modelling view of simple moving averages," International Journal of Production Research, Taylor & Francis Journals, vol. 56(18), pages 6034-6047, September.
- Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207.
- Hyndman, Rob J., 2020.
"A brief history of forecasting competitions,"
International Journal of Forecasting, Elsevier, vol. 36(1), pages 7-14.
- Rob J Hyndman, 2019. "A Brief History of Forecasting Competitions," Monash Econometrics and Business Statistics Working Papers 3/19, Monash University, Department of Econometrics and Business Statistics.
- Barrow, Devon K. & Kourentzes, Nikolaos, 2016. "Distributions of forecasting errors of forecast combinations: Implications for inventory management," International Journal of Production Economics, Elsevier, vol. 177(C), pages 24-33.
- Bojer, Casper Solheim & Meldgaard, Jens Peder, 2021. "Kaggle forecasting competitions: An overlooked learning opportunity," International Journal of Forecasting, Elsevier, vol. 37(2), pages 587-603.
- Fildes, Robert & Kolassa, Stephan & Ma, Shaohui, 2022. "Post-script—Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1319-1324.
- Kourentzes, Nikolaos & Barrow, Devon & Petropoulos, Fotios, 2019. "Another look at forecast selection and combination: Evidence from forecast pooling," International Journal of Production Economics, Elsevier, vol. 209(C), pages 226-235.
- Hong, Tao & Pinson, Pierre & Fan, Shu & Zareipour, Hamidreza & Troccoli, Alberto & Hyndman, Rob J., 2016. "Probabilistic energy forecasting: Global Energy Forecasting Competition 2014 and beyond," International Journal of Forecasting, Elsevier, vol. 32(3), pages 896-913.
- Syntetos, Aris A. & Boylan, John E., 2005. "The accuracy of intermittent demand estimates," International Journal of Forecasting, Elsevier, vol. 21(2), pages 303-314.
- R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
- Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2022. "Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1283-1318.
- Petropoulos, Fotios & Makridakis, Spyros & Assimakopoulos, Vassilios & Nikolopoulos, Konstantinos, 2014. "‘Horses for Courses’ in demand forecasting," European Journal of Operational Research, Elsevier, vol. 237(1), pages 152-163.
- Trapero, Juan R. & Cardós, Manuel & Kourentzes, Nikolaos, 2019. "Quantile forecast optimal combination to enhance safety stock estimation," International Journal of Forecasting, Elsevier, vol. 35(1), pages 239-250.
- Victor Richmond R. Jose & Robert L. Winkler, 2009. "Evaluating Quantile Assessments," Operations Research, INFORMS, vol. 57(5), pages 1287-1297, October.
- Spiliotis, Evangelos & Kouloumos, Andreas & Assimakopoulos, Vassilios & Makridakis, Spyros, 2020. "Are forecasting competitions data representative of the reality?," International Journal of Forecasting, Elsevier, vol. 36(1), pages 37-53.
- Yael Grushka-Cockayne & Victor Richmond R. Jose & Kenneth C. Lichtendahl Jr., 2017. "Ensembles of Overfit and Overconfident Forecasts," Management Science, INFORMS, vol. 63(4), pages 1110-1130, April.
- Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
- Bahman Rostami‐Tabar & M. Zied Babai & Aris Syntetos & Yves Ducq, 2013. "Demand forecasting by temporal aggregation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 60(6), pages 479-498, September.
- Robert L. Winkler & Yael Grushka-Cockayne & Kenneth C. Lichtendahl Jr. & Victor Richmond R. Jose, 2019. "Probability Forecasts and Their Combination: A Research Perspective," Decision Analysis, INFORMS, vol. 16(4), pages 239-260, December.
- Anil Gaba & Ilia Tsetlin & Robert L. Winkler, 2017. "Combining Interval Forecasts," Decision Analysis, INFORMS, vol. 14(1), pages 1-20, March.
- Nassim Nicholas Taleb & Yaneer Bar-Yam & Pasquale Cirillo, 2020. "On Single Point Forecasts for Fat-Tailed Variables," Papers 2007.16096, arXiv.org.
- Hyndman, Rob J. & Khandakar, Yeasmin, 2008.
"Automatic Time Series Forecasting: The forecast Package for R,"
Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
- Rob J. Hyndman & Yeasmin Khandakar, 2007. "Automatic time series forecasting: the forecast package for R," Monash Econometrics and Business Statistics Working Papers 6/07, Monash University, Department of Econometrics and Business Statistics.
- Hyndman, Rob J. & Ahmed, Roman A. & Athanasopoulos, George & Shang, Han Lin, 2011.
"Optimal combination forecasts for hierarchical time series,"
Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2579-2589, September.
- Rob J. Hyndman & Roman A. Ahmed & George Athanasopoulos, 2007. "Optimal combination forecasts for hierarchical time series," Monash Econometrics and Business Statistics Working Papers 9/07, Monash University, Department of Econometrics and Business Statistics.
- Fildes, Robert & Petropoulos, Fotios, 2015. "Simple versus complex selection rules for forecasting many time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1692-1701.
- Tashman, Leonard J., 2000. "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting, Elsevier, vol. 16(4), pages 437-450.
- Marshall Fisher & Ananth Raman, 1996. "Reducing the Cost of Demand Uncertainty Through Accurate Response to Early Sales," Operations Research, INFORMS, vol. 44(1), pages 87-99, February.
- Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "The M5 competition: Background, organization, and implementation," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1325-1336.
- Gneiting, Tilmann, 2011. "Quantiles as optimal point forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 197-207, April.
- Hyndman, Rob J. & Koehler, Anne B., 2006.
"Another look at measures of forecast accuracy,"
International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
- Rob J. Hyndman & Anne B. Koehler, 2005. "Another Look at Measures of Forecast Accuracy," Monash Econometrics and Business Statistics Working Papers 13/05, Monash University, Department of Econometrics and Business Statistics.
- Spiliotis, Evangelos & Makridakis, Spyros & Kaltsounis, Anastasios & Assimakopoulos, Vassilios, 2021. "Product sales probabilistic forecasting: An empirical evaluation using the M5 competition data," International Journal of Production Economics, Elsevier, vol. 240(C).
- Theodorou, Evangelos & Wang, Shengjie & Kang, Yanfei & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2022. "Exploring the representativeness of the M5 competition data," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1500-1506.
- Gardner, Everette Jr., 2006. "Exponential smoothing: The state of the art--Part II," International Journal of Forecasting, Elsevier, vol. 22(4), pages 637-666.
- Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
- Panagiotelis, Anastasios & Gamakumara, Puwasala & Athanasopoulos, George & Hyndman, Rob J., 2023.
"Probabilistic forecast reconciliation: Properties, evaluation and score optimisation,"
European Journal of Operational Research, Elsevier, vol. 306(2), pages 693-706.
- Anastasios Panagiotelis & Puwasala Gamakumara & George Athanasopoulos & Rob J Hyndman, 2020. "Probabilistic Forecast Reconciliation: Properties, Evaluation and Score Optimisation," Monash Econometrics and Business Statistics Working Papers 26/20, Monash University, Department of Econometrics and Business Statistics.
- Tilmann Gneiting, 2008. "Editorial: Probabilistic forecasting," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(2), pages 319-321, April.
- Spyros Makridakis & Evangelos Spiliotis & Vassilios Assimakopoulos, 2018. "Statistical and Machine Learning forecasting methods: Concerns and ways forward," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-26, March.
- Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
- Kolassa, Stephan, 2016. "Evaluating predictive count data distributions in retail sales forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 788-803.
- Koning, Alex J. & Franses, Philip Hans & Hibon, Michele & Stekler, H.O., 2005. "The M3 competition: Statistical tests of the results," International Journal of Forecasting, Elsevier, vol. 21(3), pages 397-409.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Olivares, Kin G. & Meetei, O. Nganba & Ma, Ruijun & Reddy, Rohan & Cao, Mengfei & Dicker, Lee, 2024. "Probabilistic hierarchical forecasting with deep Poisson mixtures," International Journal of Forecasting, Elsevier, vol. 40(2), pages 470-489.
- Wang, Shengjie & Kang, Yanfei & Petropoulos, Fotios, 2024. "Combining probabilistic forecasts of intermittent demand," European Journal of Operational Research, Elsevier, vol. 315(3), pages 1038-1048.
- Ord, J. Keith, 2022. "The uncertainty track: Machine learning, statistical modeling, synthesis," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1526-1530.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022.
"Forecasting: theory and practice,"
International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
- Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
- Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "M5 accuracy competition: Results, findings, and conclusions," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1346-1364.
- Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
- Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
- Kang, Yanfei & Spiliotis, Evangelos & Petropoulos, Fotios & Athiniotis, Nikolaos & Li, Feng & Assimakopoulos, Vassilios, 2021. "Déjà vu: A data-centric forecasting approach through time series cross-similarity," Journal of Business Research, Elsevier, vol. 132(C), pages 719-731.
- Evangelos Spiliotis & Spyros Makridakis & Artemios-Anargyros Semenoglou & Vassilios Assimakopoulos, 2022. "Comparison of statistical and machine learning methods for daily SKU demand forecasting," Operational Research, Springer, vol. 22(3), pages 3037-3061, July.
- Fotios Petropoulos & Enno Siemsen, 2023. "Forecast Selection and Representativeness," Management Science, INFORMS, vol. 69(5), pages 2672-2690, May.
- Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2022. "Predicting/hypothesizing the findings of the M5 competition," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1337-1345.
- Spiliotis, Evangelos & Petropoulos, Fotios, 2024. "On the update frequency of univariate forecasting models," European Journal of Operational Research, Elsevier, vol. 314(1), pages 111-121.
- Spiliotis, Evangelos & Assimakopoulos, Vassilios & Makridakis, Spyros, 2020. "Generalizing the Theta method for automatic forecasting," European Journal of Operational Research, Elsevier, vol. 284(2), pages 550-558.
- Spiliotis, Evangelos & Makridakis, Spyros & Kaltsounis, Anastasios & Assimakopoulos, Vassilios, 2021. "Product sales probabilistic forecasting: An empirical evaluation using the M5 competition data," International Journal of Production Economics, Elsevier, vol. 240(C).
- Kang, Yanfei & Cao, Wei & Petropoulos, Fotios & Li, Feng, 2022. "Forecast with forecasts: Diversity matters," European Journal of Operational Research, Elsevier, vol. 301(1), pages 180-190.
- Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
- Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
- Evangelos Spiliotis & Fotios Petropoulos & Vassilios Assimakopoulos, 2023. "On the Disagreement of Forecasting Model Selection Criteria," Forecasting, MDPI, vol. 5(2), pages 1-12, June.
- Jennifer L. Castle & Jurgen A. Doornik & David F. Hendry, 2021. "Forecasting Principles from Experience with Forecasting Competitions," Forecasting, MDPI, vol. 3(1), pages 1-28, February.
- Wang, Xiaoqian & Kang, Yanfei & Hyndman, Rob J. & Li, Feng, 2023.
"Distributed ARIMA models for ultra-long time series,"
International Journal of Forecasting, Elsevier, vol. 39(3), pages 1163-1184.
- Xiaoqian Wang & Yanfei Kang & Rob J Hyndman & Feng Li, 2020. "Distributed ARIMA Models for Ultra-long Time Series," Monash Econometrics and Business Statistics Working Papers 29/20, Monash University, Department of Econometrics and Business Statistics.
- Wang, Shengjie & Kang, Yanfei & Petropoulos, Fotios, 2024. "Combining probabilistic forecasts of intermittent demand," European Journal of Operational Research, Elsevier, vol. 315(3), pages 1038-1048.
- Kourentzes, Nikolaos & Trapero, Juan R. & Barrow, Devon K., 2020. "Optimising forecasting models for inventory planning," International Journal of Production Economics, Elsevier, vol. 225(C).
- Spyros Makridakis & Chris Fry & Fotios Petropoulos & Evangelos Spiliotis, 2022. "The Future of Forecasting Competitions: Design Attributes and Principles," INFORMS Joural on Data Science, INFORMS, vol. 1(1), pages 96-113, April.
More about this item
Keywords
Forecasting competitions; M competitions; Uncertainty; Probabilistic forecasts; Time series; Machine learning; Retail sales forecasting;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:38:y:2022:i:4:p:1365-1385. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.