IDEAS home Printed from https://ideas.repec.org/a/wly/jforec/v41y2022i3p455-490.html
   My bibliography  Save this article

Random forest versus logit models: Which offers better early warning of fiscal stress?

Author

Listed:
  • Barbara Jarmulska

Abstract

This study seeks to answer whether it is possible to design an effective and useful machine learning‐based early warning system signaling a risk of fiscal stress in the near future. To do so, multiple models based on econometric logit and the random forest models are designed and compared. Using a dataset of 20 annual frequency variables pertaining to 43 advanced and emerging countries during 1992–2018, the results confirm the possibility of obtaining an effective model, which correctly predicts 70–80% of fiscal stress events and tranquil periods. The random forest‐based early warning model outperforms logit models both in terms of aggregate forecasting properties and when applied to an example of the euro area sovereign debt crisis. While the very effective random forest model is commonly understood to provide lower interpretability than logit models do, this study employs tools that can be used to provide useful information for understanding what is behind the black‐box. These tools can provide information on the most important explanatory variables and on the shape of the relationship between these variables and the outcome classification. Thus, the study contributes to the discussion on the usefulness of machine learning methods in economics.

Suggested Citation

  • Barbara Jarmulska, 2022. "Random forest versus logit models: Which offers better early warning of fiscal stress?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(3), pages 455-490, April.
  • Handle: RePEc:wly:jforec:v:41:y:2022:i:3:p:455-490
    DOI: 10.1002/for.2806
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/for.2806
    Download Restriction: no

    File URL: https://libkey.io/10.1002/for.2806?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    2. Kumar, Mohan & Moorthy, Uma & Perraudin, William, 2003. "Predicting emerging market currency crashes," Journal of Empirical Finance, Elsevier, vol. 10(4), pages 427-454, September.
    3. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    4. Ciarlone, Alessio & Trebeschi, Giorgio, 2005. "Designing an early warning system for debt crises," Emerging Markets Review, Elsevier, vol. 6(4), pages 376-395, December.
    5. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    6. repec:zbw:bofitp:2011_002 is not listed on IDEAS
    7. Moritz Schularick & Alan M. Taylor, 2012. "Credit Booms Gone Bust: Monetary Policy, Leverage Cycles, and Financial Crises, 1870-2008," American Economic Review, American Economic Association, vol. 102(2), pages 1029-1061, April.
    8. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Is the 2007 US Sub-Prime Financial Crisis So Different?: An International Historical Comparison," Panoeconomicus, Savez ekonomista Vojvodine, Novi Sad, Serbia, vol. 56(3), pages 291-299.
    9. Bluwstein, Kristina & Buckmann, Marcus & Joseph, Andreas & Kapadia, Sujit & Şimşek, Özgür, 2023. "Credit growth, the yield curve and financial crisis prediction: Evidence from a machine learning approach," Journal of International Economics, Elsevier, vol. 145(C).
    10. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    11. Fioramanti, Marco, 2008. "Predicting sovereign debt crises using artificial neural networks: A comparative approach," Journal of Financial Stability, Elsevier, vol. 4(2), pages 149-164, June.
    12. Manasse, Paolo & Roubini, Nouriel, 2009. ""Rules of thumb" for sovereign debt crises," Journal of International Economics, Elsevier, vol. 78(2), pages 192-205, July.
    13. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    14. Rupa Duttagupta & Mr. Paul Cashin, 2008. "The Anatomy of Banking Crises," IMF Working Papers 2008/093, International Monetary Fund.
    15. Benedict S. Jimenez, 2017. "Institutional Constraints, Rule-Following, and Circumvention: Tax and Expenditure Limits and the Choice of Fiscal Tools During a Budget Crisis," Public Budgeting & Finance, Wiley Blackwell, vol. 37(2), pages 5-34, June.
    16. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    17. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    18. Carmen M. Reinhart & Kenneth S. Rogoff, 2011. "From Financial Crash to Debt Crisis," American Economic Review, American Economic Association, vol. 101(5), pages 1676-1706, August.
    19. Mr. Axel Schimmelpfennig & Nouriel Roubini & Paolo Manasse, 2003. "Predicting Sovereign Debt Crises," IMF Working Papers 2003/221, International Monetary Fund.
    20. Demirgüç-Kunt, Asli & Detragiache, Enrica, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, National Institute of Economic and Social Research, vol. 192, pages 68-83, April.
    21. Demyanyk, Yuliya & Hasan, Iftekhar, 2010. "Financial crises and bank failures: A review of prediction methods," Omega, Elsevier, vol. 38(5), pages 315-324, October.
    22. Fuertes, Ana-Maria & Kalotychou, Elena, 2006. "Early warning systems for sovereign debt crises: The role of heterogeneity," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1420-1441, November.
    23. Fuertes, Ana-Maria & Kalotychou, Elena, 2007. "Optimal design of early warning systems for sovereign debt crises," International Journal of Forecasting, Elsevier, vol. 23(1), pages 85-100.
    24. Juan J. Cruces & Christoph Trebesch, 2013. "Sovereign Defaults: The Price of Haircuts," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(3), pages 85-117, July.
    25. Markus Holopainen & Peter Sarlin, 2017. "Toward robust early-warning models: a horse race, ensembles and model uncertainty," Quantitative Finance, Taylor & Francis Journals, vol. 17(12), pages 1933-1963, December.
    26. David Beers & Jean-Sébastien Nadeau, 2014. "Database of Sovereign Defaults, 2015 (Revised May 2015)," Technical Reports 101, Bank of Canada.
    27. Miss Gabriela Dobrescu & Iva Petrova & Nazim Belhocine & Mr. Emanuele Baldacci, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    28. David T. Mitchell & Dean Stansel, 2016. "The Determinants of the Severity of State Fiscal Crises," Public Budgeting & Finance, Wiley Blackwell, vol. 36(4), pages 50-67, December.
    29. Barrell, Ray & Davis, E. Philip & Karim, Dilruba & Liadze, Iana, 2010. "Bank regulation, property prices and early warning systems for banking crises in OECD countries," Journal of Banking & Finance, Elsevier, vol. 34(9), pages 2255-2264, September.
    30. Davis, E. Philip & Karim, Dilruba, 2008. "Could Early Warning Systems Have Helped To Predict the Sub-Prime Crisis?," National Institute Economic Review, National Institute of Economic and Social Research, vol. 206, pages 35-47, October.
    31. Pablo Hernández de Cos & Enrique Moral-Benito & Gerrit B. Koester & Christiane Nickel, 2014. "Signalling fiscal stress in the euro area: A country-specific early warning system," Working Papers 1418, Banco de España.
    32. Davis, E. Philip & Karim, Dilruba, 2008. "Could Early Warning Systems Have Helped To Predict the Sub-Prime Crisis?," National Institute Economic Review, National Institute of Economic and Social Research, vol. 206, pages 35-47, October.
    33. Marco Lo Duca & Tuomas Peltonen, 2011. "Macrofinancial vulnerabilities and future financial stress: assessing systemic risks and predicting systemic events," BIS Papers chapters, in: Bank for International Settlements (ed.), Macroprudential regulation and policy, volume 60, pages 82-88, Bank for International Settlements.
    34. Katia Berti & Matteo Salto & Matthieu Lequien, 2012. "An early-detection index of fiscal stress for EU countries," European Economy - Economic Papers 2008 - 2015 475, Directorate General Economic and Financial Affairs (DG ECFIN), European Commission.
    35. Christoph Trebesch & Mr. Michael G. Papaioannou & Mr. Udaibir S Das, 2012. "Sovereign Debt Restructurings 1950-2010: Literature Survey, Data, and Stylized Facts," IMF Working Papers 2012/203, International Monetary Fund.
    36. Graciela Laura Kaminsky, 1999. "Currency and Banking Crises: The Early Warnings of Distress," IMF Working Papers 1999/178, International Monetary Fund.
    37. repec:zbw:bofrdp:2009_035 is not listed on IDEAS
    38. Kar‐yiu Wong & Richard Y. K. Ho, 2002. "The Asian Crisis, 1997," Review of International Economics, Wiley Blackwell, vol. 10(1), pages 1-1, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Donato Ceci & Andrea Silvestrini, 2023. "Nowcasting the state of the Italian economy: The role of financial markets," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1569-1593, November.
    2. Valencia, Oscar & Parra, Diego A. & Díaz, Juan Camilo, 2022. "Assessing Macro-Fiscal Risk for Latin American and Caribbean Countries," IDB Publications (Working Papers) 12482, Inter-American Development Bank.
    3. Raffaele Marchi & Alessandro Moro, 2024. "Forecasting Fiscal Crises in Emerging Markets and Low-Income Countries with Machine Learning Models," Open Economies Review, Springer, vol. 35(1), pages 189-213, February.
    4. Jorge M. Uribe, 2023. ""Fiscal crises and climate change"," IREA Working Papers 202303, University of Barcelona, Research Institute of Applied Economics, revised Feb 2023.
    5. Jiaming Liu & Chengzhang Li & Peng Ouyang & Jiajia Liu & Chong Wu, 2023. "Interpreting the prediction results of the tree‐based gradient boosting models for financial distress prediction with an explainable machine learning approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(5), pages 1112-1137, August.
    6. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    7. Moreno Badia, Marialuz & Medas, Paulo & Gupta, Pranav & Xiang, Yuan, 2022. "Debt is not free," Journal of International Money and Finance, Elsevier, vol. 127(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    2. Casabianca, Elizabeth Jane & Catalano, Michele & Forni, Lorenzo & Giarda, Elena & Passeri, Simone, 2022. "A machine learning approach to rank the determinants of banking crises over time and across countries," Journal of International Money and Finance, Elsevier, vol. 129(C).
    3. Moreno Badia, Marialuz & Medas, Paulo & Gupta, Pranav & Xiang, Yuan, 2022. "Debt is not free," Journal of International Money and Finance, Elsevier, vol. 127(C).
    4. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    5. Betz, Frank & Oprică, Silviu & Peltonen, Tuomas A. & Sarlin, Peter, 2014. "Predicting distress in European banks," Journal of Banking & Finance, Elsevier, vol. 45(C), pages 225-241.
    6. Mathonnat, Clément & Minea, Alexandru, 2018. "Financial development and the occurrence of banking crises," Journal of Banking & Finance, Elsevier, vol. 96(C), pages 344-354.
    7. Tamás Kristóf, 2021. "Sovereign Default Forecasting in the Era of the COVID-19 Crisis," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    8. Stijn Claessens & M. Ayhan Kose, 2013. "Financial Crises: Explanations, Types and Implications," CAMA Working Papers 2013-06, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    9. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    10. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    11. Huynh, Tran & Uebelmesser, Silke, 2024. "Early warning models for systemic banking crises: Can political indicators improve prediction?," European Journal of Political Economy, Elsevier, vol. 81(C).
    12. Hamdaoui, Mekki, 2016. "Are systemic banking crises in developed and developing countries predictable?," Journal of Multinational Financial Management, Elsevier, vol. 37, pages 114-138.
    13. Fendel Ralf & Stremmel Hanno, 2016. "Characteristics of Banking Crises: A Comparative Study with Geographical Contagion," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(3), pages 349-388, May.
    14. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    15. Rho, Caterina & Saenz, Manrique, 2021. "Financial stress and the probability of sovereign default," Journal of International Money and Finance, Elsevier, vol. 110(C).
    16. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    17. Caggiano, Giovanni & Calice, Pietro & Leonida, Leone & Kapetanios, George, 2016. "Comparing logit-based early warning systems: Does the duration of systemic banking crises matter?," Journal of Empirical Finance, Elsevier, vol. 37(C), pages 104-116.
    18. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    19. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    20. Honda, Jiro & Tapsoba, René & Issifou, Ismael, 2022. "When do we repair the roof? Insights from responses to fiscal crisis early warning signals," International Economics, Elsevier, vol. 172(C), pages 349-367.

    More about this item

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • H63 - Public Economics - - National Budget, Deficit, and Debt - - - Debt; Debt Management; Sovereign Debt
    • G01 - Financial Economics - - General - - - Financial Crises

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jforec:v:41:y:2022:i:3:p:455-490. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.