IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1608.05498.html
   My bibliography  Save this paper

Elicitability and backtesting: Perspectives for banking regulation

Author

Listed:
  • Natalia Nolde
  • Johanna F. Ziegel

Abstract

Conditional forecasts of risk measures play an important role in internal risk management of financial institutions as well as in regulatory capital calculations. In order to assess forecasting performance of a risk measurement procedure, risk measure forecasts are compared to the realized financial losses over a period of time and a statistical test of correctness of the procedure is conducted. This process is known as backtesting. Such traditional backtests are concerned with assessing some optimality property of a set of risk measure estimates. However, they are not suited to compare different risk estimation procedures. We investigate the proposal of comparative backtests, which are better suited for method comparisons on the basis of forecasting accuracy, but necessitate an elicitable risk measure. We argue that supplementing traditional backtests with comparative backtests will enhance the existing trading book regulatory framework for banks by providing the correct incentive for accuracy of risk measure forecasts. In addition, the comparative backtesting framework could be used by banks internally as well as by researchers to guide selection of forecasting methods. The discussion focuses on three risk measures, Value-at-Risk, expected shortfall and expectiles, and is supported by a simulation study and data analysis.

Suggested Citation

  • Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    • Handle: RePEc:arx:papers:1608.05498
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1608.05498
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kuan, Chung-Ming & Yeh, Jin-Huei & Hsu, Yu-Chin, 2009. "Assessing value at risk with CARE, the Conditional Autoregressive Expectile models," Journal of Econometrics, Elsevier, vol. 150(2), pages 261-270, June.
    2. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    3. Koenker,Roger, 2005. "Quantile Regression," Cambridge Books, Cambridge University Press, number 9780521845731, January.
    4. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    5. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    6. Zhu, Dongming & Galbraith, John W., 2010. "A generalized asymmetric Student-t distribution with application to financial econometrics," Journal of Econometrics, Elsevier, vol. 157(2), pages 297-305, August.
    7. Christoffersen, Peter, 2011. "Elements of Financial Risk Management," Elsevier Monographs, Elsevier, edition 2, number 9780123744487.
    8. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    9. Rama Cont & Romain Deguest & Giacomo Scandolo, 2010. "Robustness and sensitivity analysis of risk measurement procedures," Quantitative Finance, Taylor & Francis Journals, vol. 10(6), pages 593-606.
    10. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    11. Belkacem Abdous & Bruno Remillard, 1995. "Relating quantiles and expectiles under weighted-symmetry," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 47(2), pages 371-384, June.
    12. Robert F. Nau, 1985. "Should Scoring Rules be "Effective"?," Management Science, INFORMS, vol. 31(5), pages 527-535, May.
    13. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    14. Stefan Weber, 2006. "Distribution‐Invariant Risk Measures, Information, And Dynamic Consistency," Mathematical Finance, Wiley Blackwell, vol. 16(2), pages 419-441, April.
    15. So Yeon Chun & Alexander Shapiro & Stan Uryasev, 2012. "Conditional Value-at-Risk and Average Value-at-Risk: Estimation and Asymptotics," Operations Research, INFORMS, vol. 60(4), pages 739-756, August.
    16. Susanne Emmer & Marie Kratz & Dirk Tasche, 2013. "What is the best risk measure in practice? A comparison of standard measures," Papers 1312.1645, arXiv.org, revised Apr 2015.
    17. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    18. Hans Föllmer & Alexander Schied, 2002. "Convex measures of risk and trading constraints," Finance and Stochastics, Springer, vol. 6(4), pages 429-447.
    19. Thomson, William, 1979. "Eliciting production possibilities from a well-informed manager," Journal of Economic Theory, Elsevier, vol. 20(3), pages 360-380, June.
    20. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    21. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    22. Keith Kuester & Stefan Mittnik & Marc S. Paolella, 2006. "Value-at-Risk Prediction: A Comparison of Alternative Strategies," Journal of Financial Econometrics, Oxford University Press, vol. 4(1), pages 53-89.
    23. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    24. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    25. McNeil, Alexander J. & Frey, Rudiger, 2000. "Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 271-300, November.
    26. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    27. Rama Cont & Romain Deguest & Giacomo Scandolo, 2010. "Robustness and sensitivity analysis of risk measurement procedures," Post-Print hal-00413729, HAL.
    28. Fabio Bellini & Valeria Bignozzi, 2015. "On elicitable risk measures," Quantitative Finance, Taylor & Francis Journals, vol. 15(5), pages 725-733, May.
    29. Bellini, Fabio & Klar, Bernhard & Müller, Alfred & Rosazza Gianin, Emanuela, 2014. "Generalized quantiles as risk measures," Insurance: Mathematics and Economics, Elsevier, vol. 54(C), pages 41-48.
    30. Hansen, Bruce E, 1994. "Autoregressive Conditional Density Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(3), pages 705-730, August.
    31. Wang, Ruodu & Ziegel, Johanna F., 2015. "Elicitable distortion risk measures: A concise proof," Statistics & Probability Letters, Elsevier, vol. 100(C), pages 172-175.
    32. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    33. Mark H. A. Davis, 2014. "Verification of internal risk measure estimates," Papers 1410.4382, arXiv.org, revised Nov 2015.
    34. Newey, Whitney K & Powell, James L, 1987. "Asymmetric Least Squares Estimation and Testing," Econometrica, Econometric Society, vol. 55(4), pages 819-847, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steven Kou & Xianhua Peng, 2016. "On the Measurement of Economic Tail Risk," Operations Research, INFORMS, vol. 64(5), pages 1056-1072, October.
    2. Tobias Fissler & Jana Hlavinová & Birgit Rudloff, 2021. "Elicitability and identifiability of set-valued measures of systemic risk," Finance and Stochastics, Springer, vol. 25(1), pages 133-165, January.
    3. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    4. Tobias Fissler & Hajo Holzmann, 2022. "Measurability of functionals and of ideal point forecasts," Papers 2203.08635, arXiv.org.
    5. Marie Kratz & Yen H Lok & Alexander J Mcneil, 2016. "Multinomial var backtests: A simple implicit approach to backtesting expected shortfall," Working Papers hal-01424279, HAL.
    6. Tobias Fissler & Fangda Liu & Ruodu Wang & Linxiao Wei, 2024. "Elicitability and identifiability of tail risk measures," Papers 2404.14136, arXiv.org, revised Jun 2024.
    7. Kratz, Marie & Lok, Y-H & McNeil, Alexander J., 2016. "Multinomial VaR Backtests: A simple implicit approach to backtesting expected shortfall," ESSEC Working Papers WP1617, ESSEC Research Center, ESSEC Business School.
    8. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    9. Fissler Tobias & Ziegel Johanna F., 2021. "On the elicitability of range value at risk," Statistics & Risk Modeling, De Gruyter, vol. 38(1-2), pages 25-46, January.
    10. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    11. Yen, Yu-Min & Yen, Tso-Jung, 2021. "Testing forecast accuracy of expectiles and quantiles with the extremal consistent loss functions," International Journal of Forecasting, Elsevier, vol. 37(2), pages 733-758.
    12. Johanna F. Ziegel, 2013. "Coherence and elicitability," Papers 1303.1690, arXiv.org, revised Mar 2014.
    13. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    14. Enrique Molina‐Muñoz & Andrés Mora‐Valencia & Javier Perote, 2021. "Backtesting expected shortfall for world stock index ETFs with extreme value theory and Gram–Charlier mixtures," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4163-4189, July.
    15. Catania, Leopoldo & Grassi, Stefano, 2022. "Forecasting cryptocurrency volatility," International Journal of Forecasting, Elsevier, vol. 38(3), pages 878-894.
    16. Tobias Fissler & Johanna F. Ziegel, 2019. "Evaluating Range Value at Risk Forecasts," Papers 1902.04489, arXiv.org, revised Nov 2020.
    17. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    18. Mucahit Aygun & Fabio Bellini & Roger J. A. Laeven, 2023. "Elicitability of Return Risk Measures," Papers 2302.13070, arXiv.org, revised Mar 2023.
    19. Xu, Qifa & Chen, Lu & Jiang, Cuixia & Yu, Keming, 2020. "Mixed data sampling expectile regression with applications to measuring financial risk," Economic Modelling, Elsevier, vol. 91(C), pages 469-486.
    20. Dingshi Tian & Zongwu Cai & Ying Fang, 2018. "Econometric Modeling of Risk Measures: A Selective Review of the Recent Literature," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201807, University of Kansas, Department of Economics, revised Oct 2018.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1608.05498. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.