IDEAS home Printed from https://ideas.repec.org/a/taf/jnlbes/v33y2015i2p270-281.html
   My bibliography  Save this article

Evaluating the Calibration of Multi-Step-Ahead Density Forecasts Using Raw Moments

Author

Listed:
  • Malte Knüppel

Abstract

The evaluation of multi-step-ahead density forecasts is complicated by the serial correlation of the corresponding probability integral transforms. In the literature, three testing approaches can be found that take this problem into account. However, these approaches rely on data-dependent critical values, ignore important information and, therefore lack power, or suffer from size distortions even asymptotically. This article proposes a new testing approach based on raw moments. It is extremely easy to implement, uses standard critical values, can include all moments regarded as important, and has correct asymptotic size. It is found to have good size and power properties in finite samples if it is based on the (standardized) probability integral transforms.

Suggested Citation

  • Malte Knüppel, 2015. "Evaluating the Calibration of Multi-Step-Ahead Density Forecasts Using Raw Moments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 270-281, April.
    • Handle: RePEc:taf:jnlbes:v:33:y:2015:i:2:p:270-281
    DOI: 10.1080/07350015.2014.948175
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/07350015.2014.948175
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/07350015.2014.948175?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bontemps, Christian & Meddahi, Nour, 2005. "Testing normality: a GMM approach," Journal of Econometrics, Elsevier, vol. 124(1), pages 149-186, January.
    2. Kenneth F. Wallis, 2008. "Forecast Uncertainty, Its Representation And Evaluation," World Scientific Book Chapters, in: Roberto S Mariano & Yiu-Kuen Tse (ed.), Econometric Forecasting And High-Frequency Data Analysis, chapter 1, pages 1-51, World Scientific Publishing Co. Pte. Ltd..
    3. Christian Bontemps & Nour Meddahi, 2012. "Testing distributional assumptions: A GMM aproach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 978-1012, September.
    4. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    5. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    6. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    7. Michael P. Clements, 2004. "Evaluating the Bank of England Density Forecasts of Inflation," Economic Journal, Royal Economic Society, vol. 114(498), pages 844-866, October.
    8. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    9. Jushan Bai & Serena Ng, 2005. "Tests for Skewness, Kurtosis, and Normality for Time Series Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 49-60, January.
    10. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    11. Wolden Bache, Ida & Sofie Jore, Anne & Mitchell, James & Vahey, Shaun P., 2011. "Combining VAR and DSGE forecast densities," Journal of Economic Dynamics and Control, Elsevier, vol. 35(10), pages 1659-1670, October.
    12. Antonello D'Agostino & Luca Gambetti & Domenico Giannone, 2013. "Macroeconomic forecasting and structural change," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(1), pages 82-101, January.
    13. Knut Are Aastveit & Karsten R. Gerdrup & Anne Sofie Jore & Leif Anders Thorsrud, 2014. "Nowcasting GDP in Real Time: A Density Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 48-68, January.
    14. James Mitchell & Stephen G. Hall, 2005. "Evaluating, Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 995-1033, December.
    15. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    16. Kenneth F. Wallis, 2004. "An Assessment of Bank of England and National Institute Inflation Forecast Uncertainties," National Institute Economic Review, National Institute of Economic and Social Research, vol. 189(1), pages 64-71, July.
    17. Breuer, Péter & Major, Péter, 1983. "Central limit theorems for non-linear functionals of Gaussian fields," Journal of Multivariate Analysis, Elsevier, vol. 13(3), pages 425-441, September.
    18. Anthony Tay & Kenneth F. Wallis, 2000. "Density Forecasting: A Survey," Econometric Society World Congress 2000 Contributed Papers 0370, Econometric Society.
    19. Engle, Robert F. & White (the late), Halbert (ed.), 1999. "Cointegration, Causality, and Forecasting: Festschrift in Honour of Clive W. J. Granger," OUP Catalogue, Oxford University Press, number 9780198296836.
    20. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    21. Yi‐Ting Chen, 2011. "Moment tests for density forecast evaluation in the presence of parameter estimation uncertainty," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 30(4), pages 409-450, July.
    22. Jurgen A. Doornik & Henrik Hansen, 2008. "An Omnibus Test for Univariate and Multivariate Normality," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 927-939, December.
    23. James Mitchell & Kenneth F. Wallis, 2011. "Evaluating density forecasts: forecast combinations, model mixtures, calibration and sharpness," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(6), pages 1023-1040, September.
    24. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    25. Knut Are Aastveit & Karsten R. Gerdrup & Anne Sofie Jore & Leif Anders Thorsrud, 2014. "Nowcasting GDP in Real Time: A Density Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(1), pages 48-68, January.
    26. Tsyplakov, Alexander, 2011. "Evaluating density forecasts: a comment," MPRA Paper 31184, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rodríguez, Gabriel, 2017. "Modeling Latin-American stock and Forex markets volatility: Empirical application of a model with random level shifts and genuine long memory," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 393-420.
    2. Liu, Li & Wang, Yudong & Yang, Li, 2018. "Predictability of crude oil prices: An investor perspective," Energy Economics, Elsevier, vol. 75(C), pages 193-205.
    3. Gian Luigi Mazzi & James Mitchell & Gaetana Montana, 2014. "Density Nowcasts and Model Combination: Nowcasting Euro-Area GDP Growth over the 2008–09 Recession," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 76(2), pages 233-256, April.
    4. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    5. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    6. Dr. James Mitchell, 2008. "Evaluating Density Forecasts: Forecast Combinations, Model Mixtures, Calibration and Sharpness," National Institute of Economic and Social Research (NIESR) Discussion Papers 320, National Institute of Economic and Social Research.
    7. Christian Kascha & Francesco Ravazzolo, 2010. "Combining inflation density forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 231-250.
    8. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    9. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    10. Tommaso Proietti & Martyna Marczak & Gianluigi Mazzi, 2017. "Euromind‐ D : A Density Estimate of Monthly Gross Domestic Product for the Euro Area," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(3), pages 683-703, April.
    11. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2019. "Density Forecasting," BEMPS - Bozen Economics & Management Paper Series BEMPS59, Faculty of Economics and Management at the Free University of Bozen.
    12. Andres Trujillo-Barrera & Philip Garcia & Mindy L Mallory, 2018. "Short-term price density forecasts in the lean hog futures market," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 45(1), pages 121-142.
    13. Knut Are Aastveit & Francesco Ravazzolo & Herman K. van Dijk, 2018. "Combined Density Nowcasting in an Uncertain Economic Environment," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(1), pages 131-145, January.
    14. Cees Diks & Valentyn Panchenko & Dick van Dijk, 2008. "Partial Likelihood-Based Scoring Rules for Evaluating Density Forecasts in Tails," Tinbergen Institute Discussion Papers 08-050/4, Tinbergen Institute.
    15. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    16. Jonas Dovern & Hans Manner, 2020. "Order‐invariant tests for proper calibration of multivariate density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 440-456, June.
    17. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    18. Rossi, Barbara & Sekhposyan, Tatevik, 2019. "Alternative tests for correct specification of conditional predictive densities," Journal of Econometrics, Elsevier, vol. 208(2), pages 638-657.
    19. Ida Wolden Bache & James Mitchell & Francesco Ravazzolo & Shaun P. Vahey, 2009. "Macro modelling with many models," Working Paper 2009/15, Norges Bank.
    20. Anne Opschoor & Dick van Dijk & Michel van der Wel, 2014. "Improving Density Forecasts and Value-at-Risk Estimates by Combining Densities," Tinbergen Institute Discussion Papers 14-090/III, Tinbergen Institute.

    More about this item

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlbes:v:33:y:2015:i:2:p:270-281. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UBES20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.