IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/31184.html
   My bibliography  Save this paper

Evaluating density forecasts: a comment

Author

Listed:
  • Tsyplakov, Alexander

Abstract

This is a comment on Mitchell and Wallis (2011) which in turn is a critical reaction to Gneiting et al. (2007). The comment discusses the notion of forecast calibration, the advantage of using scoring rules, the “sharpness” principle and a general approach to testing calibration. The aim is to show how a more general and explicitly stated framework for evaluation of probabilistic forecasts can provide further insights.

Suggested Citation

  • Tsyplakov, Alexander, 2011. "Evaluating density forecasts: a comment," MPRA Paper 31184, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:31184
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/31184/1/MPRA_paper_31184.pdf
    File Function: original version
    Download Restriction: no
    File URL: https://mpra.ub.uni-muenchen.de/31233/1/MPRA_paper_31233.pdf
    File Function: revised version
    Download Restriction: no
    File URL: https://mpra.ub.uni-muenchen.de/32728/1/MPRA_paper_32728.pdf
    File Function: revised version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Michael P. Clements & Nick Taylor, 2003. "Evaluating interval forecasts of high-frequency financial data," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(4), pages 445-456.
    2. Valentina Corradi & Norman Swanson, 2006. "Predictive Density Evaluation. Revised," Departmental Working Papers 200621, Rutgers University, Department of Economics.
    3. Dr. James Mitchell, 2008. "Evaluating Density Forecasts: Forecast Combinations, Model Mixtures, Calibration and Sharpness," National Institute of Economic and Social Research (NIESR) Discussion Papers 320, National Institute of Economic and Social Research.
    4. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive Density Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 5, pages 197-284, Elsevier.
    5. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    6. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Taillardat, Maxime & Fougères, Anne-Laure & Naveau, Philippe & de Fondeville, Raphaël, 2023. "Evaluating probabilistic forecasts of extremes using continuous ranked probability score distributions," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1448-1459.
    2. Malte Knuppel & Fabian Kruger & Marc-Oliver Pohle, 2022. "Score-based calibration testing for multivariate forecast distributions," Papers 2211.16362, arXiv.org, revised Dec 2023.
    3. Taylor, James W., 2020. "A strategic predictive distribution for tests of probabilistic calibration," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1380-1388.
    4. Malte Knüppel, 2015. "Evaluating the Calibration of Multi-Step-Ahead Density Forecasts Using Raw Moments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 270-281, April.
    5. Tsyplakov, Alexander, 2012. "Assessment of probabilistic forecasts: Proper scoring rules and moments," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 27(3), pages 115-132.
    6. Allen, Sam & Koh, Jonathan & Segers, Johan & Ziegel, Johanna, 2024. "Tail calibration of probabilistic forecasts," LIDAM Discussion Papers ISBA 2024018, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    7. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    8. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    9. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    2. Geweke, John & Amisano, Gianni, 2011. "Optimal prediction pools," Journal of Econometrics, Elsevier, vol. 164(1), pages 130-141, September.
    3. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    4. Lenza, Michele & Moutachaker, Inès & Paredes, Joan, 2023. "Density forecasts of inflation: a quantile regression forest approach," CEPR Discussion Papers 18298, C.E.P.R. Discussion Papers.
    5. Zanetti Chini, Emilio, 2018. "Forecasting dynamically asymmetric fluctuations of the U.S. business cycle," International Journal of Forecasting, Elsevier, vol. 34(4), pages 711-732.
    6. Tsyplakov, Alexander, 2013. "Evaluation of Probabilistic Forecasts: Proper Scoring Rules and Moments," MPRA Paper 45186, University Library of Munich, Germany.
    7. Rossi, Barbara & Sekhposyan, Tatevik, 2019. "Alternative tests for correct specification of conditional predictive densities," Journal of Econometrics, Elsevier, vol. 208(2), pages 638-657.
    8. Ida Wolden Bache & James Mitchell & Francesco Ravazzolo & Shaun P. Vahey, 2009. "Macro modelling with many models," Working Paper 2009/15, Norges Bank.
    9. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    10. Emilio Zanetti Chini, 2013. "Generalizing smooth transition autoregressions," CREATES Research Papers 2013-32, Department of Economics and Business Economics, Aarhus University.
    11. Ravazzolo Francesco & Vahey Shaun P., 2014. "Forecast densities for economic aggregates from disaggregate ensembles," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(4), pages 367-381, September.
    12. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    13. John W. Galbraith & Simon van Norden, 2009. "Calibration and Resolution Diagnostics for Bank of England Density Forecasts," CIRANO Working Papers 2009s-36, CIRANO.
    14. Knut Are Aastveit & Karsten R. Gerdrup & Anne Sofie Jore & Leif Anders Thorsrud, 2014. "Nowcasting GDP in Real Time: A Density Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 48-68, January.
    15. Tsyplakov, Alexander, 2012. "Assessment of probabilistic forecasts: Proper scoring rules and moments," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 27(3), pages 115-132.
    16. Galbraith, John W. & van Norden, Simon, 2011. "Kernel-based calibration diagnostics for recession and inflation probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1041-1057, October.
    17. Reason Lesego Machete, 2011. "Early Warning with Calibrated and Sharper Probabilistic Forecasts," Papers 1112.6390, arXiv.org, revised Jan 2012.
    18. Sánchez, Ismael, 2011. "Densidad de predicción basada en momentos condicionados y máxima entropía : aplicación a la predicción de potencia eólica," DES - Working Papers. Statistics and Econometrics. WS ws111813, Universidad Carlos III de Madrid. Departamento de Estadística.
    19. Knut Are Aastveit & Karsten R. Gerdrup & Anne Sofie Jore & Leif Anders Thorsrud, 2014. "Nowcasting GDP in Real Time: A Density Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 48-68, January.
    20. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.

    More about this item

    Keywords

    density forecasts;

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:31184. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.