IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v58y2024i5d10.1007_s11135-024-01866-1.html
   My bibliography  Save this article

A spectral approach to evaluating VaR forecasts: stock market evidence from the subprime mortgage crisis, through COVID-19, to the Russo–Ukrainian war

Author

Listed:
  • Marta Małecka

    (University of Łódź
    Prague University of Economics and Business)

  • Radosław Pietrzyk

    (Wrocław University of Economics and Business)

Abstract

We explore the application of spectral methods in risk management as means of validating VaR models. We propose to replace earlier spectral VaR tests with the test based on the Anderson–Darling statistic. Based on assumptions relevant to VaR failure analysis, we experimentally prove that the Anderson–Darling spectral test displays strong power to reject inaccurate VaR. Its main advantage over the existing methods is the combination of two features: the lack of tendency to overreject properly predicted VaR and high sensitivity to limited evidence of incorrectness in VaR predictions. Thus, this test may play an important role in times of change in volatility dynamics, such as outbreaks of financial crises. We confirm this empirically, based on data starting before the subprime mortgage crisis, running through the COVID-19 pandemic, until the outbreak of the Russo–Ukrainian war. We give a number of examples when this method revealed the inaccuracy of VaR predictions not discovered by com- monly used tests. We also show that the proposed spectral test never failed at finding the models indicated as incorrect by other tests.

Suggested Citation

  • Marta Małecka & Radosław Pietrzyk, 2024. "A spectral approach to evaluating VaR forecasts: stock market evidence from the subprime mortgage crisis, through COVID-19, to the Russo–Ukrainian war," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(5), pages 4533-4567, October.
  • Handle: RePEc:spr:qualqt:v:58:y:2024:i:5:d:10.1007_s11135-024-01866-1
    DOI: 10.1007/s11135-024-01866-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-024-01866-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11135-024-01866-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bertrand Candelon & Gilbert Colletaz & Christophe Hurlin & Sessi Tokpavi, 2011. "Backtesting Value-at-Risk: A GMM Duration-Based Test," Journal of Financial Econometrics, Oxford University Press, vol. 9(2), pages 314-343, Spring.
    2. Liu, Wei & Semeyutin, Artur & Lau, Chi Keung Marco & Gozgor, Giray, 2020. "Forecasting Value-at-Risk of Cryptocurrencies with RiskMetrics type models," Research in International Business and Finance, Elsevier, vol. 54(C).
    3. Han, Xuyuan & Liu, Zhenya & Wang, Shixuan, 2022. "An R-vine copula analysis of non-ferrous metal futures with application in Value-at-Risk forecasting," Journal of Commodity Markets, Elsevier, vol. 25(C).
    4. Tubbenhauer, Tobias & Fieberg, Christian & Poddig, Thorsten, 2021. "Multi-agent-based VaR forecasting," Journal of Economic Dynamics and Control, Elsevier, vol. 131(C).
    5. Dufour, Jean-Marie, 2006. "Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics," Journal of Econometrics, Elsevier, vol. 133(2), pages 443-477, August.
    6. Zhu, Huiming & Li, Shuang & Huang, Zishan, 2023. "Frequency domain quantile dependence and connectedness between crude oil and exchange rates: Evidence from oil-importing and exporting countries," The Quarterly Review of Economics and Finance, Elsevier, vol. 90(C), pages 1-30.
    7. Lyu, Yongjian & Wang, Peng & Wei, Yu & Ke, Rui, 2017. "Forecasting the VaR of crude oil market: Do alternative distributions help?," Energy Economics, Elsevier, vol. 66(C), pages 523-534.
    8. Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
    9. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    10. Francq, Christian & Zakoïan, Jean-Michel, 2020. "Virtual Historical Simulation for estimating the conditional VaR of large portfolios," Journal of Econometrics, Elsevier, vol. 217(2), pages 356-380.
    11. Xu, Yan & Wang, Xinyu & Liu, Hening, 2021. "Quantile-based GARCH-MIDAS: Estimating value-at-risk using mixed-frequency information," Finance Research Letters, Elsevier, vol. 43(C).
    12. Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. "On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
    13. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    14. Durlauf, Steven N., 1991. "Spectral based testing of the martingale hypothesis," Journal of Econometrics, Elsevier, vol. 50(3), pages 355-376, December.
    15. Patra, Saswat, 2021. "Revisiting value-at-risk and expected shortfall in oil markets under structural breaks: The role of fat-tailed distributions," Energy Economics, Elsevier, vol. 101(C).
    16. Deo, Rohit S., 2000. "Spectral tests of the martingale hypothesis under conditional heteroscedasticity," Journal of Econometrics, Elsevier, vol. 99(2), pages 291-315, December.
    17. Denis Pelletier & Wei Wei, 2016. "The Geometric-VaR Backtesting Method," Journal of Financial Econometrics, Oxford University Press, vol. 14(4), pages 725-745.
    18. Raymond Kwong & Helen Wong, 2022. "Value-at-risk in the presence of asset price bubbles," Journal of Applied Economics, Taylor & Francis Journals, vol. 25(1), pages 361-384, December.
    19. Emmanuel Joel Aikins Abakah & Aviral Kumar Tiwari & Imhotep Paul Alagidede & Shawkat Hammoudeh, 2023. "Nonlinearity in the causality and systemic risk spillover between the OPEC oil and GCC equity markets: a pre- and post-financial crisis analysis," Empirical Economics, Springer, vol. 65(3), pages 1027-1103, September.
    20. Wu, Xinyu & Xia, Michelle & Zhang, Huanming, 2020. "Forecasting VaR using realized EGARCH model with skewness and kurtosis," Finance Research Letters, Elsevier, vol. 32(C).
    21. Mensi, Walid & El Khoury, Rim & Ali, Syed Riaz Mahmood & Vo, Xuan Vinh & Kang, Sang Hoon, 2023. "Quantile dependencies and connectedness between the gold and cryptocurrency markets: Effects of the COVID-19 crisis," Research in International Business and Finance, Elsevier, vol. 65(C).
    22. Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
    23. H. Kaibuchi & Y. Kawasaki & G. Stupfler, 2022. "GARCH-UGH: a bias-reduced approach for dynamic extreme Value-at-Risk estimation in financial time series," Quantitative Finance, Taylor & Francis Journals, vol. 22(7), pages 1277-1294, July.
    24. Kakade, Kshitij & Jain, Ishan & Mishra, Aswini Kumar, 2022. "Value-at-Risk forecasting: A hybrid ensemble learning GARCH-LSTM based approach," Resources Policy, Elsevier, vol. 78(C).
    25. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    26. Odusami, Babatunde O, 2021. "Forecasting the Value-at-Risk of REITs using realized volatility jump models," The North American Journal of Economics and Finance, Elsevier, vol. 58(C).
    27. McNeil, Alexander J. & Frey, Rudiger, 2000. "Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 271-300, November.
    28. Jiang, Kunliang & Zeng, Linhui & Song, Jiashan & Liu, Yimeng, 2022. "Forecasting Value-at-Risk of cryptocurrencies using the time-varying mixture-accelerating generalized autoregressive score model," Research in International Business and Finance, Elsevier, vol. 61(C).
    29. Chebbi, Ali & Hedhli, Amel, 2022. "Revisiting the accuracy of standard VaR methods for risk assessment: Using the Copula–EVT multidimensional approach for stock markets in the MENA region," The Quarterly Review of Economics and Finance, Elsevier, vol. 84(C), pages 430-445.
    30. Arian, Hamid & Moghimi, Mehrdad & Tabatabaei, Ehsan & Zamani, Shiva, 2022. "Encoded Value-at-Risk: A machine learning approach for portfolio risk measurement," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 202(C), pages 500-525.
    31. Kshitij Kakade & Aswini Kumar Mishra & Kshitish Ghate & Shivang Gupta, 2022. "Forecasting Commodity Market Returns Volatility: A Hybrid Ensemble Learning GARCH‐LSTM based Approach," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 29(2), pages 103-117, April.
    32. Dilip Kumar, 2020. "Value-at-Risk in the Presence of Structural Breaks Using Unbiased Extreme Value Volatility Estimator," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 18(3), pages 587-610, September.
    33. Leccadito, Arturo & Boffelli, Simona & Urga, Giovanni, 2014. "Evaluating the accuracy of value-at-risk forecasts: New multilevel tests," International Journal of Forecasting, Elsevier, vol. 30(2), pages 206-216.
    34. Seyfi, Seyed Mohammad Sina & Sharifi, Azin & Arian, Hamidreza, 2021. "Portfolio Value-at-Risk and expected-shortfall using an efficient simulation approach based on Gaussian Mixture Model," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 190(C), pages 1056-1079.
    35. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    2. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    3. Thiele, Stephen, 2019. "Detecting underestimates of risk in VaR models," Journal of Banking & Finance, Elsevier, vol. 101(C), pages 12-20.
    4. Sullivan Hu'e & Christophe Hurlin & Yang Lu, 2024. "Backtesting Expected Shortfall: Accounting for both duration and severity with bivariate orthogonal polynomials," Papers 2405.02012, arXiv.org, revised May 2024.
    5. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    6. Jean-Paul Laurent & Hassan Omidi Firouzi, 2022. "Market Risk and Volatility Weighted Historical Simulation After Basel III," Working Papers hal-03679434, HAL.
    7. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    8. Marta Małecka, 2021. "Testing for a serial correlation in VaR failures through the exponential autoregressive conditional duration model," Statistics in Transition New Series, Polish Statistical Association, vol. 22(1), pages 145-162, March.
    9. Denisa Banulescu-Radu & Christophe Hurlin & Jérémy Leymarie & Olivier Scaillet, 2021. "Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures," Management Science, INFORMS, vol. 67(9), pages 5730-5754, September.
    10. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    11. Małecka Marta, 2021. "Testing for a serial correlation in VaR failures through the exponential autoregressive conditional duration model," Statistics in Transition New Series, Statistics Poland, vol. 22(1), pages 145-162, March.
    12. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    13. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    14. Elena-Ivona Dumitrescu & Christophe Hurlin & Vinson Pham, 2012. "Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests," Finance, Presses universitaires de Grenoble, vol. 33(1), pages 79-112.
    15. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    16. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    17. Wei Kuang, 2022. "Oil tail-risk forecasts: from financial crisis to COVID-19," Risk Management, Palgrave Macmillan, vol. 24(4), pages 420-460, December.
    18. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    19. Zhi-Fu Mi & Yi-Ming Wei & Bao-Jun Tang & Rong-Gang Cong & Hao Yu & Hong Cao & Dabo Guan, 2017. "Risk assessment of oil price from static and dynamic modelling approaches," Applied Economics, Taylor & Francis Journals, vol. 49(9), pages 929-939, February.
    20. Nikola Radivojevic & Milena Cvjetkovic & Saša Stepanov, 2016. "The new hybrid value at risk approach based on the extreme value theory," Estudios de Economia, University of Chile, Department of Economics, vol. 43(1 Year 20), pages 29-52, June.

    More about this item

    Keywords

    Spectral test; Value-at-risk; VaR test; Anderson–Darling statistic; Financial crisis;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • D53 - Microeconomics - - General Equilibrium and Disequilibrium - - - Financial Markets

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:58:y:2024:i:5:d:10.1007_s11135-024-01866-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.