IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v38y2022i4p1546-1554.html
   My bibliography  Save this article

M5 competition uncertainty: Overdispersion, distributional forecasting, GAMLSS, and beyond

Author

Listed:
  • Ziel, Florian

Abstract

The M5 competition uncertainty track aims for probabilistic forecasting of sales of thousands of Walmart retail goods. We show that the M5 competition data face strong overdispersion and sporadic demand, especially zero demand. We discuss modeling issues concerning adequate probabilistic forecasting of such count data processes. Unfortunately, the majority of popular prediction methods used in the M5 competition (e.g. lightgbm and xgboost GBMs) fail to address the data characteristics, due to the considered objective functions. Distributional forecasting provides a suitable modeling approach to overcome those problems. The GAMLSS framework allows for flexible probabilistic forecasting using low-dimensional distributions. We illustrate how the GAMLSS approach can be applied to M5 competition data by modeling the location and scale parameters of various distributions, e.g. the negative binomial distribution. Finally, we discuss software packages for distributional modeling and their drawbacks, like the R package gamlss with its package extensions, and (deep) distributional forecasting libraries such as TensorFlow Probability.

Suggested Citation

  • Ziel, Florian, 2022. "M5 competition uncertainty: Overdispersion, distributional forecasting, GAMLSS, and beyond," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1546-1554.
    • Handle: RePEc:eee:intfor:v:38:y:2022:i:4:p:1546-1554
    DOI: 10.1016/j.ijforecast.2021.09.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207021001527
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2021.09.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Iv·n Fern·ndez-Val & Alfred Galichon, 2010. "Quantile and Probability Curves Without Crossing," Econometrica, Econometric Society, vol. 78(3), pages 1093-1125, May.
    2. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    3. Salinas, David & Flunkert, Valentin & Gasthaus, Jan & Januschowski, Tim, 2020. "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, Elsevier, vol. 36(3), pages 1181-1191.
    4. Ulrich, Matthias & Jahnke, Hermann & Langrock, Roland & Pesch, Robert & Senge, Robin, 2021. "Distributional regression for demand forecasting in e-grocery," European Journal of Operational Research, Elsevier, vol. 294(3), pages 831-842.
    5. repec:hal:spmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    6. Stasinopoulos, D. Mikis & Rigby, Robert A., 2007. "Generalized Additive Models for Location Scale and Shape (GAMLSS) in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 23(i07).
    7. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    8. Haben, Stephen & Giasemidis, Georgios, 2016. "A hybrid model of kernel density estimation and quantile regression for GEFCom2014 probabilistic load forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 1017-1022.
    9. Micha{l} Narajewski & Florian Ziel, 2020. "Ensemble Forecasting for Intraday Electricity Prices: Simulating Trajectories," Papers 2005.01365, arXiv.org, revised Aug 2020.
    10. Narajewski, Michał & Ziel, Florian, 2020. "Ensemble forecasting for intraday electricity prices: Simulating trajectories," Applied Energy, Elsevier, vol. 279(C).
    11. Kolassa, Stephan, 2016. "Evaluating predictive count data distributions in retail sales forecasting," International Journal of Forecasting, Elsevier, vol. 32(3), pages 788-803.
    12. R. A. Rigby & D. M. Stasinopoulos, 2005. "Generalized additive models for location, scale and shape," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 54(3), pages 507-554, June.
    13. Victor Chernozhukov & Ivan Fernandez-Val & Alfred Galichon, 2007. "Quantile and probability curves without crossing," CeMMAP working papers CWP10/07, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Simon Hirsch & Jonathan Berrisch & Florian Ziel, 2024. "Online Distributional Regression," Papers 2407.08750, arXiv.org, revised Aug 2024.
    2. Simon Hirsch & Florian Ziel, 2023. "Multivariate Simulation-based Forecasting for Intraday Power Markets: Modelling Cross-Product Price Effects," Papers 2306.13419, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Serafin, Tomasz & Marcjasz, Grzegorz & Weron, Rafał, 2022. "Trading on short-term path forecasts of intraday electricity prices," Energy Economics, Elsevier, vol. 112(C).
    2. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    3. Narajewski, Michał & Ziel, Florian, 2020. "Ensemble forecasting for intraday electricity prices: Simulating trajectories," Applied Energy, Elsevier, vol. 279(C).
    4. Nadja Klein & Michael Stanley Smith & David J. Nott, 2020. "Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices," Papers 2010.01844, arXiv.org, revised May 2021.
    5. Maike Hohberg & Peter Pütz & Thomas Kneib, 2020. "Treatment effects beyond the mean using distributional regression: Methods and guidance," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-29, February.
    6. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    7. Micha{l} Narajewski, 2022. "Probabilistic forecasting of German electricity imbalance prices," Papers 2205.11439, arXiv.org.
    8. Micha{l} Narajewski & Florian Ziel, 2020. "Ensemble Forecasting for Intraday Electricity Prices: Simulating Trajectories," Papers 2005.01365, arXiv.org, revised Aug 2020.
    9. Christopher Kath & Florian Ziel, 2020. "Optimal Order Execution in Intraday Markets: Minimizing Costs in Trade Trajectories," Papers 2009.07892, arXiv.org, revised Oct 2020.
    10. Nadja Klein & Michael Stanley Smith & David J. Nott, 2023. "Deep distributional time series models and the probabilistic forecasting of intraday electricity prices," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 493-511, June.
    11. Adam Bee & Joshua Mitchell & Nikolas Mittag & Jonathan Rothbaum & Carl Sanders & Lawrence Schmidt & Matthew Unrath, 2023. "National Experimental Wellbeing Statistics - Version 1," Working Papers 23-04, Center for Economic Studies, U.S. Census Bureau.
    12. Boyao Zhang & Tobias Hepp & Sonja Greven & Elisabeth Bergherr, 2022. "Adaptive step-length selection in gradient boosting for Gaussian location and scale models," Computational Statistics, Springer, vol. 37(5), pages 2295-2332, November.
    13. Simon Hirsch & Florian Ziel, 2023. "Multivariate Simulation-based Forecasting for Intraday Power Markets: Modelling Cross-Product Price Effects," Papers 2306.13419, arXiv.org.
    14. Michał Narajewski, 2022. "Probabilistic Forecasting of German Electricity Imbalance Prices," Energies, MDPI, vol. 15(14), pages 1-17, July.
    15. Angela Noufaily & M. C. Jones, 2013. "Parametric quantile regression based on the generalized gamma distribution," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 62(5), pages 723-740, November.
    16. Yixuan Wang & Jianzhu Li & Ping Feng & Rong Hu, 2015. "A Time-Dependent Drought Index for Non-Stationary Precipitation Series," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 29(15), pages 5631-5647, December.
    17. Panayi, Efstathios & Peters, Gareth W. & Danielsson, Jon & Zigrand, Jean-Pierre, 2018. "Designating market maker behaviour in limit order book markets," Econometrics and Statistics, Elsevier, vol. 5(C), pages 20-44.
    18. Gauss Cordeiro & Josemar Rodrigues & Mário Castro, 2012. "The exponential COM-Poisson distribution," Statistical Papers, Springer, vol. 53(3), pages 653-664, August.
    19. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    20. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.

    More about this item

    Keywords

    M5 competition; Probabilistic forecasting; GAMLSS; Distribution modeling; Overdispersion; Count data; Demand forecasting;
    All these keywords.

    JEL classification:

    • M5 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:38:y:2022:i:4:p:1546-1554. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.