IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v28y2019i1d10.1007_s10260-018-00434-w.html
   My bibliography  Save this article

Backtesting VaR and expectiles with realized scores

Author

Listed:
  • Fabio Bellini

    (University of Milano-Bicocca)

  • Ilia Negri

    (University of Bergamo)

  • Mariya Pyatkova

    (University of Milano-Bicocca)

Abstract

Several statistical functionals such as quantiles and expectiles arise naturally as the minimizers of the expected value of a scoring function, a property that is called elicitability (see Gneiting in J Am Stat Assoc 106:746–762, 2011 and the references therein). The existence of such scoring functions gives a natural way to compare the accuracy of different forecasting models, and to test comparative hypotheses by means of the Diebold–Mariano test as suggested in a recent work. In this paper we suggest a procedure to test the accuracy of a quantile or expectile forecasting model in an absolute sense, as in the original Basel I backtesting procedure of value-at-risk. To this aim, we study the asymptotic and finite-sample distributions of empirical scores for normal and uniform i.i.d. samples. We compare on simulated data the empirical power of our procedure with alternative procedures based on empirical identification functions (i.e. in the case of VaR the number of violations) and we find an higher power in detecting at least misspecification in the mean. We conclude with a real data example where both backtesting procedures are applied to AR(1)–Garch(1,1) models fitted to SP500 logreturns for VaR and expectiles’ forecasts.

Suggested Citation

  • Fabio Bellini & Ilia Negri & Mariya Pyatkova, 2019. "Backtesting VaR and expectiles with realized scores," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(1), pages 119-142, March.
    • Handle: RePEc:spr:stmapp:v:28:y:2019:i:1:d:10.1007_s10260-018-00434-w
    DOI: 10.1007/s10260-018-00434-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10260-018-00434-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10260-018-00434-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    2. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    3. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    4. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    5. Fabio Bellini & Elena Di Bernardino, 2017. "Risk management with expectiles," The European Journal of Finance, Taylor & Francis Journals, vol. 23(6), pages 487-506, May.
    6. Stefan Weber, 2006. "Distribution‐Invariant Risk Measures, Information, And Dynamic Consistency," Mathematical Finance, Wiley Blackwell, vol. 16(2), pages 419-441, April.
    7. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    8. Thomson, William, 1979. "Eliciting production possibilities from a well-informed manager," Journal of Economic Theory, Elsevier, vol. 20(3), pages 360-380, June.
    9. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    10. Wong, Woon K., 2010. "Backtesting value-at-risk based on tail losses," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 526-538, June.
    11. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    12. McNeil, Alexander J. & Frey, Rudiger, 2000. "Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 271-300, November.
    13. Kerkhof, Jeroen & Melenberg, Bertrand, 2004. "Backtesting for risk-based regulatory capital," Journal of Banking & Finance, Elsevier, vol. 28(8), pages 1845-1865, August.
    14. Philippe Artzner & Freddy Delbaen & Jean‐Marc Eber & David Heath, 1999. "Coherent Measures of Risk," Mathematical Finance, Wiley Blackwell, vol. 9(3), pages 203-228, July.
    15. Newey, Whitney K & Powell, James L, 1987. "Asymmetric Least Squares Estimation and Testing," Econometrica, Econometric Society, vol. 55(4), pages 819-847, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriele Canna & Francesca Centrone & Emanuela Rosazza Gianin, 2021. "Capital Allocation Rules and the No-Undercut Property," Mathematics, MDPI, vol. 9(2), pages 1-13, January.
    2. Alexander Wagner & Stan Uryasev, 2019. "Portfolio Optimization with Expectile and Omega Functions," Papers 1910.14005, arXiv.org.
    3. Mustapha Rachdi & Ali Laksaci & Noriah M. Al-Kandari, 2022. "Expectile regression for spatial functional data analysis (sFDA)," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(5), pages 627-655, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steven Kou & Xianhua Peng, 2016. "On the Measurement of Economic Tail Risk," Operations Research, INFORMS, vol. 64(5), pages 1056-1072, October.
    2. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    3. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    4. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    5. Marie Kratz & Yen H Lok & Alexander J Mcneil, 2016. "Multinomial var backtests: A simple implicit approach to backtesting expected shortfall," Working Papers hal-01424279, HAL.
    6. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    7. Soren Bettels & Sojung Kim & Stefan Weber, 2022. "Multinomial Backtesting of Distortion Risk Measures," Papers 2201.06319, arXiv.org, revised Aug 2024.
    8. Kratz, Marie & Lok, Y-H & McNeil, Alexander J., 2016. "Multinomial VaR Backtests: A simple implicit approach to backtesting expected shortfall," ESSEC Working Papers WP1617, ESSEC Research Center, ESSEC Business School.
    9. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    10. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    11. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    12. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    13. Tobias Fissler & Jana Hlavinová & Birgit Rudloff, 2021. "Elicitability and identifiability of set-valued measures of systemic risk," Finance and Stochastics, Springer, vol. 25(1), pages 133-165, January.
    14. Zaichao Du & Juan Carlos Escanciano, 2017. "Backtesting Expected Shortfall: Accounting for Tail Risk," Management Science, INFORMS, vol. 63(4), pages 940-958, April.
    15. Richard Gerlach & Declan Walpole & Chao Wang, 2017. "Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility," Quantitative Finance, Taylor & Francis Journals, vol. 17(2), pages 199-215, February.
    16. Enrique Molina‐Muñoz & Andrés Mora‐Valencia & Javier Perote, 2021. "Backtesting expected shortfall for world stock index ETFs with extreme value theory and Gram–Charlier mixtures," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4163-4189, July.
    17. Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
    18. Julia S. Mehlitz & Benjamin R. Auer, 2021. "Time‐varying dynamics of expected shortfall in commodity futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 41(6), pages 895-925, June.
    19. Radu Tunaru, 2015. "Model Risk in Financial Markets:From Financial Engineering to Risk Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9524, September.
    20. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:28:y:2019:i:1:d:10.1007_s10260-018-00434-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.