IDEAS home Printed from https://ideas.repec.org/a/kap/compec/v64y2024i6d10.1007_s10614-024-10571-y.html
   My bibliography  Save this article

Evaluating Density Forecasts Using Weighted Multivariate Scores in a Risk Management Context

Author

Listed:
  • Jie Cheng

    (Keele University)

Abstract

Scoring rules are commonly applied to assess the accuracy of density forecasts in both univariate and multivariate settings. In a financial risk management context, we are mostly interested in a particular region of the density: the (left) tail of a portfolio’s return distribution. The dependence structure between returns on different assets (associated with a given portfolio) is usually time-varying and asymmetric. In this paper, we conduct a simulation study to compare the discrimination ability between the well-established scores and their threshold-weighted versions with selected regions. This facilitates a comprehensive comparison of the performance of scoring rules in different settings. Our empirical applications also confirm the importance of weighted-threshold scores for accurate estimates of Value-at-risk and related measures of downside risk.

Suggested Citation

  • Jie Cheng, 2024. "Evaluating Density Forecasts Using Weighted Multivariate Scores in a Risk Management Context," Computational Economics, Springer;Society for Computational Economics, vol. 64(6), pages 3617-3643, December.
    • Handle: RePEc:kap:compec:v:64:y:2024:i:6:d:10.1007_s10614-024-10571-y
    DOI: 10.1007/s10614-024-10571-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10614-024-10571-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10614-024-10571-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Diks, Cees & Panchenko, Valentyn & Sokolinskiy, Oleg & van Dijk, Dick, 2014. "Comparing the accuracy of multivariate density forecasts in selected regions of the copula support," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 79-94.
    2. Giacomini, Raffaella & Komunjer, Ivana, 2005. "Evaluation and Combination of Conditional Quantile Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 416-431, October.
    3. Christoffersen, Peter & Errunza, Vihang & Jacobs, Kris & Jin, Xisong, 2014. "Correlation dynamics and international diversification benefits," International Journal of Forecasting, Elsevier, vol. 30(3), pages 807-824.
    4. Aas, Kjersti & Czado, Claudia & Frigessi, Arnoldo & Bakken, Henrik, 2009. "Pair-copula constructions of multiple dependence," Insurance: Mathematics and Economics, Elsevier, vol. 44(2), pages 182-198, April.
    5. Andrew J. Patton, 2004. "On the Out-of-Sample Importance of Skewness and Asymmetric Dependence for Asset Allocation," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 130-168.
    6. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    7. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    8. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    9. Pierre Giot & Sébastien Laurent, 2003. "Value-at-risk for long and short trading positions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(6), pages 641-663.
    10. François Longin & Bruno Solnik, 2001. "Extreme Correlation of International Equity Markets," Journal of Finance, American Finance Association, vol. 56(2), pages 649-676, April.
    11. James E. Matheson & Robert L. Winkler, 1976. "Scoring Rules for Continuous Probability Distributions," Management Science, INFORMS, vol. 22(10), pages 1087-1096, June.
    12. Diks, Cees & Fang, Hao, 2020. "Comparing density forecasts in a risk management context," International Journal of Forecasting, Elsevier, vol. 36(2), pages 531-551.
    13. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    14. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    15. Jie Cheng, 2023. "Modelling and forecasting risk dependence and portfolio VaR for cryptocurrencies," Empirical Economics, Springer, vol. 65(2), pages 899-924, August.
    16. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    17. Patton, Andrew J., 2012. "A review of copula models for economic time series," Journal of Multivariate Analysis, Elsevier, vol. 110(C), pages 4-18.
    18. Han Lin Shang & Yang Yang & Fearghal Kearney, 2019. "Intraday forecasts of a volatility index: functional time series methods with dynamic updating," Annals of Operations Research, Springer, vol. 282(1), pages 331-354, November.
    19. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    20. Stephen Hora & Erim Kardeş, 2015. "Calibration, sharpness and the weighting of experts in a linear opinion pool," Annals of Operations Research, Springer, vol. 229(1), pages 429-450, June.
    21. Robert Engle, 2001. "GARCH 101: The Use of ARCH/GARCH Models in Applied Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 157-168, Fall.
    22. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    23. Nagler, T. & Bumann, C. & Czado, C., 2019. "Model selection in sparse high-dimensional vine copula models with an application to portfolio risk," Journal of Multivariate Analysis, Elsevier, vol. 172(C), pages 180-192.
    24. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    25. Drew Creal & Siem Jan Koopman & André Lucas, 2013. "Generalized Autoregressive Score Models With Applications," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(5), pages 777-795, August.
    26. Anne Opschoor & André Lucas & István Barra & Dick van Dijk, 2021. "Closed-Form Multi-Factor Copula Models With Observation-Driven Dynamic Factor Loadings," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(4), pages 1066-1079, October.
    27. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    28. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    29. James Mitchell & Stephen G. Hall, 2005. "Evaluating, Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 995-1033, December.
    30. Ang, Andrew & Chen, Joseph, 2002. "Asymmetric correlations of equity portfolios," Journal of Financial Economics, Elsevier, vol. 63(3), pages 443-494, March.
    31. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    32. Tse, Y K & Tsui, Albert K C, 2002. "A Multivariate Generalized Autoregressive Conditional Heteroscedasticity Model with Time-Varying Correlations," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 351-362, July.
    33. Yanqin Fan & Andrew J. Patton, 2014. "Copulas in Econometrics," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 179-200, August.
    34. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    35. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-350, July.
    36. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Diks, Cees & Fang, Hao, 2020. "Comparing density forecasts in a risk management context," International Journal of Forecasting, Elsevier, vol. 36(2), pages 531-551.
    2. C. Alexander & M. Coulon & Y. Han & X. Meng, 2024. "Evaluating the discrimination ability of proper multi-variate scoring rules," Annals of Operations Research, Springer, vol. 334(1), pages 857-883, March.
    3. Alexander, Carol & Han, Yang & Meng, Xiaochun, 2023. "Static and dynamic models for multivariate distribution forecasts: Proper scoring rule tests of factor-quantile versus multivariate GARCH models," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1078-1096.
    4. Anne Opschoor & Dick van Dijk & Michel van der Wel, 2014. "Improving Density Forecasts and Value-at-Risk Estimates by Combining Densities," Tinbergen Institute Discussion Papers 14-090/III, Tinbergen Institute.
    5. Jie Cheng, 2023. "Modelling and forecasting risk dependence and portfolio VaR for cryptocurrencies," Empirical Economics, Springer, vol. 65(2), pages 899-924, August.
    6. Anne Opschoor & André Lucas & István Barra & Dick van Dijk, 2021. "Closed-Form Multi-Factor Copula Models With Observation-Driven Dynamic Factor Loadings," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(4), pages 1066-1079, October.
    7. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    8. Patton, Andrew, 2013. "Copula Methods for Forecasting Multivariate Time Series," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 899-960, Elsevier.
    9. Diks, Cees & Panchenko, Valentyn & Sokolinskiy, Oleg & van Dijk, Dick, 2014. "Comparing the accuracy of multivariate density forecasts in selected regions of the copula support," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 79-94.
    10. Cees Diks & Valentyn Panchenko & Oleg Sokolinskiy, & Dick van Dijk, 2013. "Comparing the Accuracy of Copula-Based Multivariate Density Forecasts in Selected Regions of Support," Tinbergen Institute Discussion Papers 13-061/III, Tinbergen Institute.
    11. Clements, Michael P., 2018. "Are macroeconomic density forecasts informative?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 181-198.
    12. Luisa Bisaglia & Matteo Grigoletto, 2018. "A new time-varying model for forecasting long-memory series," Papers 1812.07295, arXiv.org.
    13. Marco J. Lombardi & Francesco Ravazzolo, 2012. "Oil price density forecasts: exploring the linkages with stock markets," Working Paper 2012/24, Norges Bank.
    14. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    15. Andrew Martinez, 2017. "Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter," Working Papers (Old Series) 1717, Federal Reserve Bank of Cleveland.
    16. Berg, Tim O. & Henzel, Steffen R., 2015. "Point and density forecasts for the euro area using Bayesian VARs," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1067-1095.
    17. Pierluigi Vallarino, 2024. "Dynamic kernel models," Tinbergen Institute Discussion Papers 24-082/III, Tinbergen Institute.
    18. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    19. Manzan, Sebastiano & Zerom, Dawit, 2013. "Are macroeconomic variables useful for forecasting the distribution of U.S. inflation?," International Journal of Forecasting, Elsevier, vol. 29(3), pages 469-478.
    20. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.

    More about this item

    Keywords

    Density forecast evaluation; Copula; Weighted score; Multivariate forecasting; Multivariate scoring rule; Asymmetric dependence structure;
    All these keywords.

    JEL classification:

    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:compec:v:64:y:2024:i:6:d:10.1007_s10614-024-10571-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.