IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/112079.html
   My bibliography  Save this paper

Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data

Author

Listed:
  • Yildirim, Yusuf
  • Sanyal, Anirban

Abstract

Early warning indicators (EWIs) of banking crises should ideally be judged on how well they function in relation to the choice issue faced by macroprudential policymakers. Several practical features of this challenge are translated into statistical evaluation criteria, including difficulties measuring the costs and advantages of various policy interventions, as well as requirements for the timeliness and stability of EWIs. We analyze the balance panel of possible EWIs for six countries that have experienced currency crisis and banking crisis in recent times. Using possible early warning indicators, we evaluate the suitability of these EWIs in view of their predictive power and stability of signals. The paper observes that credit disbursements to non-financial sectors and central government provides stable signal about systemic risks. Further debt service ratio, inter bank rates and total reserves are also found to be useful in predicting these crisis. Lastly, the paper observes that linear combination of these indicators improves the predictive power of EWIs further.

Suggested Citation

  • Yildirim, Yusuf & Sanyal, Anirban, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," MPRA Paper 112079, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:112079
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/112079/1/MPRA_paper_112079.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Carmen M. Reinhart & Kenneth S. Rogoff, 2014. "This Time is Different: A Panoramic View of Eight Centuries of Financial Crises," Annals of Economics and Finance, Society for AEF, vol. 15(2), pages 215-268, November.
    3. Andrew Berg & Eduardo Borensztein & Catherine Pattillo, 2005. "Assessing Early Warning Systems: How Have They Worked in Practice?," IMF Staff Papers, Palgrave Macmillan, vol. 52(3), pages 1-5.
    4. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    5. Jahn, Nadya & Kick, Thomas, 2012. "Early warning indicators for the German banking system: A macroprudential analysis," Discussion Papers 27/2012, Deutsche Bundesbank.
    6. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Preface," MPRA Paper 17451, University Library of Munich, Germany.
    7. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    8. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters, in: This Time Is Different: Eight Centuries of Financial Folly, Princeton University Press.
    9. Baker, Stuart G. & Kramer, Barnett S., 2007. "Peirce, Youden, and Receiver Operating Characteristic Curves," The American Statistician, American Statistical Association, vol. 61, pages 343-346, November.
    10. Christian Dembiermont & Mathias Drehmann & Siriporn Muksakunratana, 2013. "How much does the private sector really borrow - a new database for total credit to the private non-financial sector," BIS Quarterly Review, Bank for International Settlements, March.
    11. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Chapter 1," MPRA Paper 17452, University Library of Munich, Germany.
    12. Özgür Orhangazi, 2014. "Capital Flows and Credit Expansions in Turkey," Review of Radical Political Economics, Union for Radical Political Economics, vol. 46(4), pages 509-516, December.
    13. Òscar Jordà & Moritz Schularick & Alan M Taylor, 2011. "Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 59(2), pages 340-378, June.
    14. Mark David Witte, 2012. "What Effect Did the Credit Crisis of 2008 Have on European Exchange Rates?," Eastern European Economics, Taylor & Francis Journals, vol. 50(3), pages 79-93, May.
    15. Yu, Wenbao & Park, Taesung, 2015. "Two simple algorithms on linear combination of multiple biomarkers to maximize partial area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 15-27.
    16. Jan Babecký & Tomáš Havránek & Jakub Matìjù & Marek Rusnák & Kateøina Šmídková & Boøek Vašíèek, 2011. "Early Warning Indicators of Crisis Incidence: Evidence from a Panel of 40 Developed Countries," Working Papers IES 2011/36, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Nov 2011.
    17. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    18. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.
    19. Ms. Sally Chen & Katsiaryna Svirydzenka, 2021. "Financial Cycles – Early Warning Indicators of Banking Crises?," IMF Working Papers 2021/116, International Monetary Fund.
    20. Joon-Ho Hahm & Hyun Song Shin & Kwanho Shin, 2013. "Noncore Bank Liabilities and Financial Vulnerability," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45, pages 3-36, August.
    21. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    22. Mathias Drehmann & Mikael Juselius, 2012. "Do debt service costs affect macroeconomic and financial stability?," BIS Quarterly Review, Bank for International Settlements, September.
    23. Dornbusch, Rudiger, 1976. "Expectations and Exchange Rate Dynamics," Journal of Political Economy, University of Chicago Press, vol. 84(6), pages 1161-1176, December.
    24. Pierre-Olivier Gourinchas & Maurice Obstfeld, 2012. "Stories of the Twentieth Century for the Twenty-First," American Economic Journal: Macroeconomics, American Economic Association, vol. 4(1), pages 226-265, January.
    25. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    26. Ilja Viktorov & Alexander Abramov, 2020. "The 2014–15 Financial Crisis in Russia and the Foundations of Weak Monetary Power Autonomy in the International Political Economy," New Political Economy, Taylor & Francis Journals, vol. 25(4), pages 487-510, June.
    27. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.
    28. Martínez, Juan Francisco & Oda, Daniel, 2021. "Characterization of the Chilean financial cycle, early warning indicators and implications for macro-prudential policies," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 2(1).
    29. Simone Romano, 2021. "The 2011 Crisis in Italy: A Story of Deep-Rooted (and Still Unresolved) Economic and Political Weaknesses," Springer Books, in: Bettina De Souza Guilherme & Christian Ghymers & Stephany Griffith-Jones & Andrea Ribeiro Hoffmann (ed.), Financial Crisis Management and Democracy, edition 1, chapter 0, pages 173-184, Springer.
    30. Cohen, Jacqueline & Garman, Samuel & Gorr, Wilpen, 2009. "Empirical calibration of time series monitoring methods using receiver operating characteristic curves," International Journal of Forecasting, Elsevier, vol. 25(3), pages 484-497, July.
    31. Travis J. Berge & Òscar Jordà, 2011. "Evaluating the Classification of Economic Activity into Recessions and Expansions," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(2), pages 246-277, April.
    32. Mathias Drehmann & Claudio Borio & Leonardo Gambacorta & Gabriel Jiminez & Carlos Trucharte, 2010. "Countercyclical capital buffers: exploring options," BIS Working Papers 317, Bank for International Settlements.
    33. Berg, Andrew & Pattillo, Catherine, 1999. "Predicting currency crises:: The indicators approach and an alternative," Journal of International Money and Finance, Elsevier, vol. 18(4), pages 561-586, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    3. Jon Danielsson & Marcela Valenzuela & Ilknur Zer, 2018. "Learning from History: Volatility and Financial Crises," The Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2774-2805.
    4. Tihana Skrinjaric, 2023. "Leading indicators of financial stress in Croatia: a regime switching approach," Public Sector Economics, Institute of Public Finance, vol. 47(2), pages 205-232.
    5. Ebrahimi Kahou, Mahdi & Lehar, Alfred, 2017. "Macroprudential policy: A review," Journal of Financial Stability, Elsevier, vol. 29(C), pages 92-105.
    6. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.
    7. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.
    8. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    9. Gertler, Pavel & Hofmann, Boris, 2018. "Monetary facts revisited," Journal of International Money and Finance, Elsevier, vol. 86(C), pages 154-170.
    10. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    11. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    12. Bordo, M.D. & Meissner, C.M., 2016. "Fiscal and Financial Crises," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 355-412, Elsevier.
    13. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    14. Rakesh Padhan & K. P. Prabheesh, 2019. "Effectiveness Of Early Warning Models: A Critical Review And New Agenda For Future Direction," Bulletin of Monetary Economics and Banking, Bank Indonesia, vol. 22(4), pages 457-484, December.
    15. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    16. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    17. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
    18. Schudel, Willem, 2015. "Shifting horizons: assessing macro trends before, during, and following systemic banking crises," Working Paper Series 1766, European Central Bank.
    19. Lainà, Patrizio & Nyholm, Juho & Sarlin, Peter, 2015. "Leading indicators of systemic banking crises: Finland in a panel of EU countries," Review of Financial Economics, Elsevier, vol. 24(C), pages 18-35.
    20. Detken, Carsten & Weeken, Olaf & Alessi, Lucia & Bonfim, Diana & Boucinha, Miguel & Castro, Christian & Frontczak, Sebastian & Giordana, Gaston & Giese, Julia & Wildmann, Nadya & Kakes, Jan & Klaus, B, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 5, European Systemic Risk Board.

    More about this item

    Keywords

    EWIs; ROC; area under the curve; shrinkage;
    All these keywords.

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • G01 - Financial Economics - - General - - - Financial Crises
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:112079. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.