IDEAS home Printed from https://ideas.repec.org/p/han/dpaper/dp-529.html
   My bibliography  Save this paper

Model Risk in Backtesting Risk Measures

Author

Listed:
  • Evers, Corinna
  • Rohde, Johannes

Abstract

Under the Basel II regulatory framework non-negligible statistical problems arise when backtesting risk measures. In this setting backtests often become infeasible due to a low number of violations leading to heavy size distortions. According to Escanciano and Olmo (2010, 2011) these problems persist when incorporating estimation and model risk by adjusting the asymptotic variance of the test statistics. In this paper, we analyze backtests based on hit and duration sequences in a univariate framework by running a simulation study in order to identify the problems of backtests that examine the adequacy of Value at Risk measures. One main finding indicates that backtests of all classes show heavy size distortions. These problems for the relevant Basel II set-up, however, cannot be alleviated by modifying backtests in a way that accounts for estimation risk or misspecification risk.

Suggested Citation

  • Evers, Corinna & Rohde, Johannes, 2014. "Model Risk in Backtesting Risk Measures," Hannover Economic Papers (HEP) dp-529, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät.
    • Handle: RePEc:han:dpaper:dp-529
    as

    Download full text from publisher

    File URL: http://diskussionspapiere.wiwi.uni-hannover.de/pdf_bib/dp-529.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bertrand Candelon & Gilbert Colletaz & Christophe Hurlin & Sessi Tokpavi, 2011. "Backtesting Value-at-Risk: A GMM Duration-Based Test," Journal of Financial Econometrics, Oxford University Press, vol. 9(2), pages 314-343, Spring.
    2. Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (U.S.).
    3. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    4. Russell, Jeffrey & Engle, Robert F, 1998. "Econometric Analysis of Discrete-Valued Irregularly-Spaced Financial Transactions Data Using a New Autoregressive Conditional Multinomial Model," University of California at San Diego, Economics Working Paper Series qt00m2c5hk, Department of Economics, UC San Diego.
    5. Escanciano, J. C. & Olmo, J., 2007. "Estimation risk effects on backtesting for parametric value-at-risk models," Working Papers 07/11, Department of Economics, City University London.
    6. Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
    7. Cristina Danciulescu, 2010. "Backtesting Value-at-Risk Models: A Multivariate Approach," Caepr Working Papers 2010-004, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
    8. Cristina Danciulescu, 2010. "Backtesting Value-at-Risk Models: A Multivariate Approach," CAEPR Working Papers 2010-004, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    9. J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Oxford University Press, vol. 9(1), pages 132-161, Winter.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    2. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    3. Thiele, Stephen, 2019. "Detecting underestimates of risk in VaR models," Journal of Banking & Finance, Elsevier, vol. 101(C), pages 12-20.
    4. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    5. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    6. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    7. Jean-Paul Laurent & Hassan Omidi Firouzi, 2022. "Market Risk and Volatility Weighted Historical Simulation After Basel III," Working Papers hal-03679434, HAL.
    8. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    9. Bertrand Candelon & Gilbert Colletaz & Christophe Hurlin & Sessi Tokpavi, 2011. "Backtesting Value-at-Risk: A GMM Duration-Based Test," Journal of Financial Econometrics, Oxford University Press, vol. 9(2), pages 314-343, Spring.
    10. Sullivan Hu'e & Christophe Hurlin & Yang Lu, 2024. "Backtesting Expected Shortfall: Accounting for both duration and severity with bivariate orthogonal polynomials," Papers 2405.02012, arXiv.org, revised May 2024.
    11. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    12. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    13. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    14. Elena-Ivona Dumitrescu & Christophe Hurlin & Vinson Pham, 2012. "Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests," Finance, Presses universitaires de Grenoble, vol. 33(1), pages 79-112.
    15. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    16. Marta Małecka & Radosław Pietrzyk, 2024. "A spectral approach to evaluating VaR forecasts: stock market evidence from the subprime mortgage crisis, through COVID-19, to the Russo–Ukrainian war," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(5), pages 4533-4567, October.
    17. Yannick Hoga & Matei Demetrescu, 2023. "Monitoring Value-at-Risk and Expected Shortfall Forecasts," Management Science, INFORMS, vol. 69(5), pages 2954-2971, May.
    18. Juan Carlos Escanciano & Zaichao Du, 2015. "Backtesting Expected Shortfall: Accounting for Tail Risk," CAEPR Working Papers 2015-001, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    19. Christophe Hurlin & Christophe Pérignon, 2012. "Margin Backtesting," Working Papers halshs-00746274, HAL.
    20. Timo Dimitriadis & iaochun Liu & Julie Schnaitmann, 2023. "Encompassing Tests for Value at Risk and Expected Shortfall Multistep Forecasts Based on Inference on the Boundary," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 412-444.

    More about this item

    Keywords

    Model risk; backtesting; Value at risk;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • G32 - Financial Economics - - Corporate Finance and Governance - - - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:han:dpaper:dp-529. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Heidrich, Christian (email available below). General contact details of provider: https://edirc.repec.org/data/fwhande.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.