IDEAS home Printed from https://ideas.repec.org/a/cai/finpug/fina_331_0079.html
   My bibliography  Save this article

Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests

Author

Listed:
  • Elena-Ivona Dumitrescu
  • Christophe Hurlin
  • Vinson Pham

Abstract

In this paper we propose a new tool for backtesting that examines the quality of Value-at-Risk (VaR) forecasts. To date, the most distinguished regression-based backtest, proposed by Engle and Manganelli (2004), relies on a linear model. However, in view of the dichotomic character of the series of violations, a non-linear model seems more appropriate. In this paper we thus propose a new tool for backtesting (denoted DB) based on a dynamic binary regression model. Our discrete-choice model, e.g. Probit, Logit, links the sequence of violations to a set of explanatory variables including the lagged VaR and the lagged violations in particular. It allows us to separately test the unconditional coverage, the independence and the conditional coverage hypotheses and it is easy to implement. Monte-Carlo experiments show that the DB test exhibits good small sample properties in realistic sample settings (5% coverage rate with estimation risk). An application on a portfolio composed of three assets included in the CAC40 market index is finally proposed.

Suggested Citation

  • Elena-Ivona Dumitrescu & Christophe Hurlin & Vinson Pham, 2012. "Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests," Finance, Presses universitaires de Grenoble, vol. 33(1), pages 79-112.
  • Handle: RePEc:cai:finpug:fina_331_0079
    as

    Download full text from publisher

    File URL: http://www.cairn.info/load_pdf.php?ID_ARTICLE=FINA_331_0079
    Download Restriction: free

    File URL: http://www.cairn.info/revue-finance-2012-1-page-79.htm
    Download Restriction: free
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christophe Pérignon & R.D. Smith, 2008. "A New Approach to Comparing VaR Estimation Methods," Post-Print hal-00854087, HAL.
    2. Bertrand Candelon & Gilbert Colletaz & Christophe Hurlin & Sessi Tokpavi, 2011. "Backtesting Value-at-Risk: A GMM Duration-Based Test," Journal of Financial Econometrics, Oxford University Press, vol. 9(2), pages 314-343, Spring.
    3. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value‐at‐Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    4. Gaglianone, Wagner Piazza & Lima, Luiz Renato & Linton, Oliver & Smith, Daniel R., 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 150-160.
    5. Christophe HURLIN & Sessi TOKPAVI, 2007. "Une évaluation des procédures de Backtesting," LEO Working Papers / DR LEO 1716, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    6. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    7. Dufour, Jean-Marie, 2006. "Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics," Journal of Econometrics, Elsevier, vol. 133(2), pages 443-477, August.
    8. Engle, Robert F & Manganelli, Simone, 1999. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," University of California at San Diego, Economics Working Paper Series qt06m3d6nv, Department of Economics, UC San Diego.
    9. Gourieroux, C. & Laurent, J. P. & Scaillet, O., 2000. "Sensitivity analysis of Values at Risk," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 225-245, November.
    10. Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
    11. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    12. Christophe Hurlin & Sessi Tokpavi, 2007. "Un test de validité de la Value at Risk," Revue économique, Presses de Sciences-Po, vol. 58(3), pages 599-608.
    13. Engle, Robert F & Sheppard, Kevin K, 2001. "Theoretical and Empirical Properties of Dynamic Conditional Correlation Multivariate GARCH," University of California at San Diego, Economics Working Paper Series qt5s2218dp, Department of Economics, UC San Diego.
    14. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    15. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-350, July.
    16. Heikki Kauppi & Pentti Saikkonen, 2008. "Predicting U.S. Recessions with Dynamic Binary Response Models," The Review of Economics and Statistics, MIT Press, vol. 90(4), pages 777-791, November.
    17. Elisabetta Falcetti & Merxe Tudela, 2006. "Modelling Currency Crises in Emerging Markets: A Dynamic Probit Model with Unobserved Heterogeneity and Autocorrelated Errors," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 68(4), pages 445-471, August.
    18. Christophe Hurlin & Sessi Tokpavi, 2008. "Une évaluation des procédures de Backtesting. « Tout va pour le mieux dans le meilleur des mondes »," Finance, Presses universitaires de Grenoble, vol. 29(1), pages 53-80.
    19. Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
    20. Gourieroux,Christian, 2000. "Econometrics of Qualitative Dependent Variables," Cambridge Books, Cambridge University Press, number 9780521589857.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rachida Hennani, 2015. "Can the Lasota(1977)’s model compete with the Mackey-Glass(1977)’s model in nonlinear modelling of financial time series?," Working Papers 15-09, LAMETA, Universtiy of Montpellier, revised Jun 2015.
    2. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    3. Emrah Altun & Huseyin Tatlidil & Gamze Ozel & Saralees Nadarajah, 2018. "Does the Assumption on Innovation Process Play an Important Role for Filtered Historical Simulation Model?," JRFM, MDPI, vol. 11(1), pages 1-13, January.
    4. Lyu, Yongjian & Wang, Peng & Wei, Yu & Ke, Rui, 2017. "Forecasting the VaR of crude oil market: Do alternative distributions help?," Energy Economics, Elsevier, vol. 66(C), pages 523-534.
    5. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    6. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    7. Tihana Skrinjaric & Maja Sabol, 2024. "Easier Said than Done: Predicting Downside Risks to House Prices in Croatia," Czech Journal of Economics and Finance (Finance a uver), Charles University Prague, Faculty of Social Sciences, vol. 74(1), pages 43-72, March.
    8. Jean-Paul Laurent & Hassan Omidi Firouzi, 2022. "Market Risk and Volatility Weighted Historical Simulation After Basel III," Working Papers hal-03679434, HAL.
    9. Lyu, Yongjian & Qin, Fanshu & Ke, Rui & Wei, Yu & Kong, Mengzhen, 2024. "Does mixed frequency variables help to forecast value at risk in the crude oil market?," Resources Policy, Elsevier, vol. 88(C).
    10. Lin, Weidong & Taamouti, Abderrahim, 2024. "Portfolio selection under non-gaussianity and systemic risk: A machine learning based forecasting approach," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1179-1188.
    11. Denis Pelletier & Wei Wei, 2016. "The Geometric-VaR Backtesting Method," Journal of Financial Econometrics, Oxford University Press, vol. 14(4), pages 725-745.
    12. Katerina Rigana & Ernst C. Wit & Samantha Cook, 2024. "Navigating Market Turbulence: Insights from Causal Network Contagion Value at Risk," Papers 2402.06032, arXiv.org.
    13. Ming Qi & Jing Xu & Nnenna Bridget Amuji & Shumingrui Wang & Fengqian Xu & Huan Zhou, 2022. "The Nexus among Energy Consumption, Economic Growth and Trade Openness: Evidence from West Africa," Sustainability, MDPI, vol. 14(6), pages 1-22, March.
    14. Ana-Maria Fuertes & Jose Olmo, 2016. "On Setting Day-Ahead Equity Trading Risk Limits: VaR Prediction at Market Close or Open?," JRFM, MDPI, vol. 9(3), pages 1-20, September.
    15. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    16. Tihana Škrinjarić, 2023. "Macroprudential stance assessment: problems of measurement, literature review and some comments for the case of Croatia," Working Papers 72, The Croatian National Bank, Croatia.
    17. Manuela Braione & Nicolas K. Scholtes, 2016. "Forecasting Value-at-Risk under Different Distributional Assumptions," Econometrics, MDPI, vol. 4(1), pages 1-27, January.
    18. Uddin, Gazi Salah & Hernandez, Jose Areola & Shahzad, Syed Jawad Hussain & Hedström, Axel, 2018. "Multivariate dependence and spillover effects across energy commodities and diversification potentials of carbon assets," Energy Economics, Elsevier, vol. 71(C), pages 35-46.
    19. repec:dau:papers:123456789/15232 is not listed on IDEAS
    20. Ziggel, Daniel & Berens, Tobias & Weiß, Gregor N.F. & Wied, Dominik, 2014. "A new set of improved Value-at-Risk backtests," Journal of Banking & Finance, Elsevier, vol. 48(C), pages 29-41.
    21. Lyu, Yongjian & Qin, Fanshu & Ke, Rui & Yang, Mo & Chang, Jianing, 2024. "Forecasting the VaR of the crude oil market: A combination of mixed data sampling and extreme value theory," Energy Economics, Elsevier, vol. 133(C).
    22. Katja Ahoniemi & Ana-Maria Fuertes & Jose Olmo, 2016. "Overnight News and Daily Equity Trading Risk Limits," Journal of Financial Econometrics, Oxford University Press, vol. 14(3), pages 525-551.
    23. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    24. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elena-Ivona DUMITRESCU, 2011. "Backesting Value-at-Risk: From DQ (Dynamic Quantile) to DB (Dynamic Binary) Tests," LEO Working Papers / DR LEO 262, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    2. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    3. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    4. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    5. Thiele, Stephen, 2019. "Detecting underestimates of risk in VaR models," Journal of Banking & Finance, Elsevier, vol. 101(C), pages 12-20.
    6. Sullivan Hu'e & Christophe Hurlin & Yang Lu, 2024. "Backtesting Expected Shortfall: Accounting for both duration and severity with bivariate orthogonal polynomials," Papers 2405.02012, arXiv.org, revised May 2024.
    7. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    8. Colletaz, Gilbert & Hurlin, Christophe & Pérignon, Christophe, 2013. "The Risk Map: A new tool for validating risk models," Journal of Banking & Finance, Elsevier, vol. 37(10), pages 3843-3854.
    9. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    10. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    11. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    12. Leccadito, Arturo & Boffelli, Simona & Urga, Giovanni, 2014. "Evaluating the accuracy of value-at-risk forecasts: New multilevel tests," International Journal of Forecasting, Elsevier, vol. 30(2), pages 206-216.
    13. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    14. Christophe Hurlin & Christophe Pérignon, 2012. "Margin Backtesting," Working Papers halshs-00746274, HAL.
    15. repec:dau:papers:123456789/15232 is not listed on IDEAS
    16. Aramonte, Sirio & Giudice Rodriguez, Marius del & Wu, Jason, 2013. "Dynamic factor Value-at-Risk for large heteroskedastic portfolios," Journal of Banking & Finance, Elsevier, vol. 37(11), pages 4299-4309.
    17. Erik Kole & Thijs Markwat & Anne Opschoor & Dick van Dijk, 2017. "Forecasting Value-at-Risk under Temporal and Portfolio Aggregation," Journal of Financial Econometrics, Oxford University Press, vol. 15(4), pages 649-677.
    18. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    19. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    20. Yannick Hoga & Matei Demetrescu, 2023. "Monitoring Value-at-Risk and Expected Shortfall Forecasts," Management Science, INFORMS, vol. 69(5), pages 2954-2971, May.
    21. Ziggel, Daniel & Berens, Tobias & Weiß, Gregor N.F. & Wied, Dominik, 2014. "A new set of improved Value-at-Risk backtests," Journal of Banking & Finance, Elsevier, vol. 48(C), pages 29-41.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cai:finpug:fina_331_0079. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jean-Baptiste de Vathaire (email available below). General contact details of provider: https://www.cairn.info/revue-finance.htm .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.