IDEAS home Printed from https://ideas.repec.org/a/eee/finsta/v45y2019ics1572308918305801.html
   My bibliography  Save this article

Does machine learning help us predict banking crises?

Author

Listed:
  • Beutel, Johannes
  • List, Sophia
  • von Schweinitz, Gregor

Abstract

This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance metric, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly efficiently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.

Suggested Citation

  • Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
  • Handle: RePEc:eee:finsta:v:45:y:2019:i:c:s1572308918305801
    DOI: 10.1016/j.jfs.2019.100693
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1572308918305801
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jfs.2019.100693?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Duprey, Thibaut & Klaus, Benjamin & Peltonen, Tuomas, 2017. "Dating systemic financial stress episodes in the EU countries," Journal of Financial Stability, Elsevier, vol. 32(C), pages 30-56.
    2. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    3. McFadden, Daniel L., 1984. "Econometric analysis of qualitative response models," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 24, pages 1395-1457, Elsevier.
    4. Charles P. Kindleberger & Robert Z. Aliber, 2005. "Manias, Panics and Crashes," Palgrave Macmillan Books, Palgrave Macmillan, edition 0, number 978-0-230-62804-5, September.
    5. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    6. El-Shagi, M. & Knedlik, T. & von Schweinitz, G., 2013. "Predicting financial crises: The (statistical) significance of the signals approach," Journal of International Money and Finance, Elsevier, vol. 35(C), pages 76-103.
    7. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Is the 2007 US Sub-Prime Financial Crisis So Different?: An International Historical Comparison," Panoeconomicus, Savez ekonomista Vojvodine, Novi Sad, Serbia, vol. 56(3), pages 291-299.
    8. Luc Laeven & Fabian Valencia, 2020. "Systemic Banking Crises Database II," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 68(2), pages 307-361, June.
    9. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    10. Schüler, Yves Stephan & Hiebert, Paul P. & Peltonen, Tuomas A., 2015. "Characterising the financial cycle: A multivariate and time-varying approach," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 112985, Verein für Socialpolitik / German Economic Association.
    11. Carmen M. Reinhart & Kenneth S. Rogoff, 2014. "This Time is Different: A Panoramic View of Eight Centuries of Financial Crises," Annals of Economics and Finance, Society for AEF, vol. 15(2), pages 215-268, November.
    12. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency Crashes in Emerging Markets: Empirical Indicators," Center for International and Development Economics Research (CIDER) Working Papers 233424, University of California-Berkeley, Department of Economics.
    13. Sarlin, Peter, 2013. "On policymakers’ loss functions and the evaluation of early warning systems," Economics Letters, Elsevier, vol. 119(1), pages 1-7.
    14. A. Berg & C. Pattillo, 1999. "What Caused the Asian Crises: An Early Warning System Approach," Economic Notes, Banca Monte dei Paschi di Siena SpA, vol. 28(3), pages 285-334, November.
    15. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    16. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    17. Sarlin, Peter & von Schweinitz, Gregor, 2021. "Optimizing Policymakers’ Loss Functions In Crisis Prediction: Before, Within Or After?," Macroeconomic Dynamics, Cambridge University Press, vol. 25(1), pages 100-123, January.
    18. Pierre-Olivier Gourinchas & Maurice Obstfeld, 2012. "Stories of the Twentieth Century for the Twenty-First," American Economic Journal: Macroeconomics, American Economic Association, vol. 4(1), pages 226-265, January.
    19. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    20. Kaminsky, Graciela L. & Reinhart, Carmen M., 2000. "On crises, contagion, and confusion," Journal of International Economics, Elsevier, vol. 51(1), pages 145-168, June.
    21. Lo Duca, Marco & Koban, Anne & Basten, Marisa & Bengtsson, Elias & Klaus, Benjamin & Kusmierczyk, Piotr & Lang, Jan Hannes & Detken, Carsten & Peltonen, Tuomas, 2017. "A new database for financial crises in European countries," ESRB Occasional Paper Series 13, European Systemic Risk Board.
    22. Brunnermeier, Markus K. & Oehmke, Martin, 2013. "Bubbles, Financial Crises, and Systemic Risk," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1221-1288, Elsevier.
    23. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters, in: This Time Is Different: Eight Centuries of Financial Folly, Princeton University Press.
    24. Katsuyuki Tanaka & Takuji Kinkyo & Shigeyuki Hamori, 2018. "Financial Hazard Map: Financial Vulnerability Predicted by a Random Forests Classification Model," Sustainability, MDPI, vol. 10(5), pages 1-18, May.
    25. repec:zbw:bofrdp:2017_012 is not listed on IDEAS
    26. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    27. Òscar Jordà & Moritz Schularick & Alan M Taylor, 2011. "Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 59(2), pages 340-378, June.
    28. Diebold, Francis X & Rudebusch, Glenn D, 1989. "Scoring the Leading Indicators," The Journal of Business, University of Chicago Press, vol. 62(3), pages 369-391, July.
    29. Markus K. Brunnermeier, 2009. "Deciphering the Liquidity and Credit Crunch 2007-2008," Journal of Economic Perspectives, American Economic Association, vol. 23(1), pages 77-100, Winter.
    30. Makram El-Shagi & Axel Lindner & Gregor von Schweinitz, 2016. "Real Effective Exchange Rate Misalignment in the Euro Area: A Counterfactual Analysis," Review of International Economics, Wiley Blackwell, vol. 24(1), pages 37-66, February.
    31. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    32. Russell Davidson & James MacKinnon, 2000. "Bootstrap tests: how many bootstraps?," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 55-68.
    33. Cerutti, Eugenio & Claessens, Stijn & Laeven, Luc, 2017. "The use and effectiveness of macroprudential policies: New evidence," Journal of Financial Stability, Elsevier, vol. 28(C), pages 203-224.
    34. International Monetary Fund, 2011. "Macroprudential Policy: What Instruments and How to Use them? Lessons From Country Experiences," IMF Working Papers 2011/238, International Monetary Fund.
    35. Markus Holopainen & Peter Sarlin, 2017. "Toward robust early-warning models: a horse race, ensembles and model uncertainty," Quantitative Finance, Taylor & Francis Journals, vol. 17(12), pages 1933-1963, December.
    36. Angela Maddaloni & Jose-Luis Peydro, 2011. "Bank Risk-taking, Securitization, Supervision, and Low Interest Rates: Evidence from the Euro-area and the U.S. Lending Standards," The Review of Financial Studies, Society for Financial Studies, vol. 24(6), pages 2121-2165.
    37. Guillermo A. Calvo, 1998. "CAPITAL FLOWS AND CAPITAL-MARKET CRISES: The Simple Economics of Sudden Stops," Journal of Applied Economics, Taylor & Francis Journals, vol. 1(1), pages 35-54, November.
    38. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency crashes in emerging markets: An empirical treatment," Journal of International Economics, Elsevier, vol. 41(3-4), pages 351-366, November.
    39. Bussiere, Matthieu & Fratzscher, Marcel, 2006. "Towards a new early warning system of financial crises," Journal of International Money and Finance, Elsevier, vol. 25(6), pages 953-973, October.
    40. Stijn Claessens, 2015. "An Overview of Macroprudential Policy Tools," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 397-422, December.
    41. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2012. "Characterising the financial cycle: don't lose sight of the medium term!," BIS Working Papers 380, Bank for International Settlements.
    42. Frédéric Boissay & Fabrice Collard & Frank Smets, 2016. "Booms and Banking Crises," Journal of Political Economy, University of Chicago Press, vol. 124(2), pages 489-538.
    43. Tanaka, Katsuyuki & Kinkyo, Takuji & Hamori, Shigeyuki, 2016. "Random forests-based early warning system for bank failures," Economics Letters, Elsevier, vol. 148(C), pages 118-121.
    44. Mathias Drehmann & Mikael Juselius & Anton Korinek, 2017. "Accounting for debt service: the painful legacy of credit booms," BIS Working Papers 645, Bank for International Settlements.
    45. Raghuram G. Rajan, 2006. "Has Finance Made the World Riskier?," European Financial Management, European Financial Management Association, vol. 12(4), pages 499-533, September.
    46. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.
    2. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    3. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    4. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    5. Tran Huynh & Silke Uebelmesser, 2022. "Early warning models for systemic banking crises: can political indicators improve prediction?," Jena Economics Research Papers 2022-007, Friedrich-Schiller-University Jena.
    6. Lo Duca, Marco & Koban, Anne & Basten, Marisa & Bengtsson, Elias & Klaus, Benjamin & Kusmierczyk, Piotr & Lang, Jan Hannes & Detken, Carsten & Peltonen, Tuomas, 2017. "A new database for financial crises in European countries," ESRB Occasional Paper Series 13, European Systemic Risk Board.
    7. Dieckelmann, Daniel, 2020. "Cross-border lending and the international transmission of banking crises," Discussion Papers 2020/13, Free University Berlin, School of Business & Economics.
    8. Casabianca, Elizabeth Jane & Catalano, Michele & Forni, Lorenzo & Giarda, Elena & Passeri, Simone, 2022. "A machine learning approach to rank the determinants of banking crises over time and across countries," Journal of International Money and Finance, Elsevier, vol. 129(C).
    9. Bordo, M.D. & Meissner, C.M., 2016. "Fiscal and Financial Crises," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 355-412, Elsevier.
    10. Sarlin, Peter & von Schweinitz, Gregor, 2021. "Optimizing Policymakers’ Loss Functions In Crisis Prediction: Before, Within Or After?," Macroeconomic Dynamics, Cambridge University Press, vol. 25(1), pages 100-123, January.
    11. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    12. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    13. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    14. Duprey, Thibaut & Klaus, Benjamin, 2022. "Early warning or too late? A (pseudo-)real-time identification of leading indicators of financial stress," Journal of Banking & Finance, Elsevier, vol. 138(C).
    15. Roy, Saktinil & Kemme, David M., 2012. "Causes of banking crises: Deregulation, credit booms and asset bubbles, then and now," International Review of Economics & Finance, Elsevier, vol. 24(C), pages 270-294.
    16. Cesa-Bianchi, Ambrogio & Eguren Martin, Fernando & Thwaites, Gregory, 2019. "Foreign booms, domestic busts: The global dimension of banking crises," Journal of Financial Intermediation, Elsevier, vol. 37(C), pages 58-74.
    17. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    18. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    19. Irma Alonso & Luis Molina, 2019. "The SHERLOC: an EWS-based index of vulnerability for emerging economies," Working Papers 1946, Banco de España.
    20. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.

    More about this item

    Keywords

    Early warning system; Logit; Machine learning; Systemic banking crises;
    All these keywords.

    JEL classification:

    • C35 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G01 - Financial Economics - - General - - - Financial Crises

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finsta:v:45:y:2019:i:c:s1572308918305801. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jfstabil .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.