IDEAS home Printed from https://ideas.repec.org/p/fip/fedgfe/2005-21.html
   My bibliography  Save this paper

A review of backtesting and backtesting procedures

Author

Listed:
  • Sean D. Campbell

Abstract

This paper reviews a variety of backtests that examine the adequacy of Value-at-Risk (VaR) measures. These backtesting procedures are reviewed from both a statistical and risk management perspective. The properties of unconditional coverage and independence are defined and their relation to backtesting procedures is discussed. Backtests are then classified by whether they examine the unconditional coverage property, independence property, or both properties of a VaR measure. Backtests that examine the accuracy of a VaR model at several quantiles, rather than a single quantile, are also outlined and discussed. The statistical power properties of these tests are examined in a simulation experiment. Finally, backtests that are specified in terms of a pre-specified loss function are reviewed and their use in VaR validation is discussed.

Suggested Citation

  • Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (U.S.).
    • Handle: RePEc:fip:fedgfe:2005-21
    as

    Download full text from publisher

    File URL: http://www.federalreserve.gov/pubs/feds/2005/200521/200521abs.html
    Download Restriction: no
    File URL: http://www.federalreserve.gov/pubs/feds/2005/200521/200521pap.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value‐at‐Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    2. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    3. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    4. Jose A. Lopez, 1996. "Regulatory Evaluation of Value-at-Risk Models," Center for Financial Institutions Working Papers 96-51, Wharton School Center for Financial Institutions, University of Pennsylvania.
    5. Matthew Pritsker, 1997. "Evaluating Value at Risk Methodologies: Accuracy versus Computational Time," Journal of Financial Services Research, Springer;Western Finance Association, vol. 12(2), pages 201-242, October.
    6. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    7. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    8. Katerina Simons, 1996. "Value at risk: new approaches to risk management," New England Economic Review, Federal Reserve Bank of Boston, issue Sep, pages 3-13.
    9. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    10. Colleen Cassidy & Marianne Gizycki, 1997. "Measuring Traded Market Risk: Value-at-risk and Backtesting Techniques," RBA Research Discussion Papers rdp9708, Reserve Bank of Australia.
    11. Matthew Pritsker, 2001. "The hidden dangers of historical simulation," Finance and Economics Discussion Series 2001-27, Board of Governors of the Federal Reserve System (U.S.).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kulp-Tåg, Sofie, 2007. "An Empirical Investigation of Value-at-Risk in Long and Short Trading Positions," Working Papers 526, Hanken School of Economics.
    2. Jean-Francois Carpantier, 2010. "Commodities inventory effect," Working Papers hal-01821158, HAL.
    3. Ergün, A. Tolga & Jun, Jongbyung, 2010. "Time-varying higher-order conditional moments and forecasting intraday VaR and Expected Shortfall," The Quarterly Review of Economics and Finance, Elsevier, vol. 50(3), pages 264-272, August.
    4. Chong Zhang & Xinyi Liu & Zhongmou Zhang & Mingyu Jin & Lingyao Li & Zhenting Wang & Wenyue Hua & Dong Shu & Suiyuan Zhu & Xiaobo Jin & Sujian Li & Mengnan Du & Yongfeng Zhang, 2024. "When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments," Papers 2407.18957, arXiv.org, revised Sep 2024.
    5. Kimera Naradh & Retius Chifurira & Knowledge Chinhamu, 2022. "Analysis of stock exchange risk and currency in South African Financial Markets using stable parameter estimation," International Journal of Finance & Banking Studies, Center for the Strategic Studies in Business and Finance, vol. 11(1), pages 120-131, January.
    6. Durán Santomil, Pablo & Otero González, Luís & Martorell Cunill, Onofre & Merigó Lindahl, José M., 2018. "Backtesting an equity risk model under Solvency II," Journal of Business Research, Elsevier, vol. 89(C), pages 216-222.
    7. CARPANTIER, Jean-François & DUFAYS, Arnaud, 2012. "Commodities volatility and the theory of storage," LIDAM Discussion Papers CORE 2012037, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    8. Lööf, Hans & Stephan, Andreas, 2019. "The Impact of ESG on Stocks’ Downside Risk and Risk Adjusted Return," Working Paper Series in Economics and Institutions of Innovation 477, Royal Institute of Technology, CESIS - Centre of Excellence for Science and Innovation Studies.
    9. Wagner Piazza Gaglianone & Luiz Renato Lima & Oliver Linton & Daniel R. Smith, 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 150-160, January.
    10. Pérignon, Christophe & Deng, Zi Yin & Wang, Zhi Jun, 2008. "Do banks overstate their Value-at-Risk?," Journal of Banking & Finance, Elsevier, vol. 32(5), pages 783-794, May.
    11. Sabrina Khanniche, 2009. "Evaluation of Hedge Fund Returns Value at Risk Using GARCH Models," EconomiX Working Papers 2009-46, University of Paris Nanterre, EconomiX.
    12. Fries, Christian P. & Nigbur, Tobias & Seeger, Norman, 2017. "Displaced relative changes in historical simulation: Application to risk measures of interest rates with phases of negative rates," Journal of Empirical Finance, Elsevier, vol. 42(C), pages 175-198.
    13. Felipe de Oliveira & Sinézio Fernandes Maia, 2017. "Volatility Forecasting before the Subprime Crisis," EcoMod2017 10376, EcoMod.
    14. Codrut Florin Ivascu & Daniela Serban, 2023. "Value at Risk Estimation for Non-Gaussian Distributions," The Review of Finance and Banking, Academia de Studii Economice din Bucuresti, Romania / Facultatea de Finante, Asigurari, Banci si Burse de Valori / Catedra de Finante, vol. 15(2), pages 181-190, December.
    15. Vêlayoudom Marimoutou & Bechir Raggad & Abdelwahed Trabelsi, 2006. "Extreme Value Theory and Value at Risk : Application to Oil Market," Working Papers halshs-00410746, HAL.
    16. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    17. Gonzales-Martínez, Rolando, 2009. "La Gestión de Riesgo de Liquidez en Economías Emergentes: Un Modelo Valor-en-Riesgo (VaR) Paramétrico de Calibración Indirecta y una Aplicación al Sistema Financiero Boliviano [Liquidity Risk Manag," MPRA Paper 14247, University Library of Munich, Germany.
    18. Christos Agiakloglou & Charalampos Agiropoulos, 2011. "The sensitivity of Value-at-Risk estimates using Monte Carlo approach," SPOUDAI Journal of Economics and Business, SPOUDAI Journal of Economics and Business, University of Piraeus, vol. 61(1-2), pages 7-12, January -.
    19. Zhang, Hanyu & Dufour, Alfonso, 2024. "Managing portfolio risk during crisis times: A dynamic conditional correlation perspective," The Quarterly Review of Economics and Finance, Elsevier, vol. 94(C), pages 241-251.
    20. Evers, Corinna & Rohde, Johannes, 2014. "Model Risk in Backtesting Risk Measures," Hannover Economic Papers (HEP) dp-529, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät.
    21. Marimoutou, Velayoudoum & Raggad, Bechir & Trabelsi, Abdelwahed, 2009. "Extreme Value Theory and Value at Risk: Application to oil market," Energy Economics, Elsevier, vol. 31(4), pages 519-530, July.
    22. Christophe HURLIN & Sessi TOKPAVI, 2006. "Backtesting VaR Accuracy: A Simple and Powerful Test," LEO Working Papers / DR LEO 268, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    23. Simon Fritzsch & Maike Timphus & Gregor Weiss, 2021. "Marginals Versus Copulas: Which Account For More Model Risk In Multivariate Risk Forecasting?," Papers 2109.10946, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    2. Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
    3. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    4. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    5. Christophe HURLIN & Sessi TOKPAVI, 2007. "Une évaluation des procédures de Backtesting," LEO Working Papers / DR LEO 1716, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    6. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 15, pages 777-878, Elsevier.
    7. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    8. Gaglianone, Wagner Piazza & Lima, Luiz Renato & Linton, Oliver & Smith, Daniel R., 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 150-160.
    9. Jose A. Lopez, 1999. "Methods for evaluating value-at-risk estimates," Economic Review, Federal Reserve Bank of San Francisco, pages 3-17.
    10. Colletaz, Gilbert & Hurlin, Christophe & Pérignon, Christophe, 2013. "The Risk Map: A new tool for validating risk models," Journal of Banking & Finance, Elsevier, vol. 37(10), pages 3843-3854.
    11. Marie Kratz & Yen H Lok & Alexander J Mcneil, 2016. "Multinomial var backtests: A simple implicit approach to backtesting expected shortfall," Working Papers hal-01424279, HAL.
    12. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    13. Pérignon, Christophe & Deng, Zi Yin & Wang, Zhi Jun, 2008. "Do banks overstate their Value-at-Risk?," Journal of Banking & Finance, Elsevier, vol. 32(5), pages 783-794, May.
    14. Farkas, Walter & Fringuellotti, Fulvia & Tunaru, Radu, 2020. "A cost-benefit analysis of capital requirements adjusted for model risk," Journal of Corporate Finance, Elsevier, vol. 65(C).
    15. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    16. Radu Tunaru, 2015. "Model Risk in Financial Markets:From Financial Engineering to Risk Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9524, August.
    17. Leccadito, Arturo & Boffelli, Simona & Urga, Giovanni, 2014. "Evaluating the accuracy of value-at-risk forecasts: New multilevel tests," International Journal of Forecasting, Elsevier, vol. 30(2), pages 206-216.
    18. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    19. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    20. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.

    More about this item

    Keywords

    Risk management; Bank investments;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgfe:2005-21. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ryan Wolfslayer ; Keisha Fournillier (email available below). General contact details of provider: https://edirc.repec.org/data/frbgvus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.