IDEAS home Printed from https://ideas.repec.org/a/gam/jjrfmx/v17y2024i2p50-d1328960.html
   My bibliography  Save this article

Accuracy Comparison between Five Machine Learning Algorithms for Financial Risk Evaluation

Author

Listed:
  • Haokun Dong

    (Faculty of Science and Technology, University of Canberra, Canberra 2617, Australia)

  • Rui Liu

    (Faculty of Science and Technology, University of Canberra, Canberra 2617, Australia)

  • Allan W. Tham

    (Faculty of Science and Technology, University of Canberra, Canberra 2617, Australia)

Abstract

An accurate prediction of loan default is crucial in credit risk evaluation. A slight deviation from true accuracy can often cause financial losses to lending institutes. This study describes the non-parametric approach that compares five different machine learning classifiers combined with a focus on sufficiently large datasets. It presents the findings on various standard performance measures such as accuracy, precision, recall and F1 scores in addition to Receiver Operating Curve-Area Under Curve (ROC-AUC). In this study, various data pre-processing techniques including normalization and standardization, imputation of missing values and the handling of imbalanced data using SMOTE will be discussed and implemented. Also, the study examines the use of hyper-parameters in various classifiers. During the model construction phase, various pipelines feed data to the five machine learning classifiers, and the performance results obtained from the five machine learning classifiers are based on sampling with SMOTE or hyper-parameters versus without SMOTE and hyper-parameters. Each classifier is compared to another in terms of accuracy during training and prediction phase based on out-of-sample data. The 2 data sets used for this experiment contain 1000 and 30,000 observations, respectively, of which the training/testing ratio is 80:20. The comparative results show that random forest outperforms the other four classifiers both in training and actual prediction.

Suggested Citation

  • Haokun Dong & Rui Liu & Allan W. Tham, 2024. "Accuracy Comparison between Five Machine Learning Algorithms for Financial Risk Evaluation," JRFM, MDPI, vol. 17(2), pages 1-19, January.
  • Handle: RePEc:gam:jjrfmx:v:17:y:2024:i:2:p:50-:d:1328960
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1911-8074/17/2/50/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1911-8074/17/2/50/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Deakin, Eb, 1972. "Discriminant Analysis Of Predictors Of Business Failure," Journal of Accounting Research, Wiley Blackwell, vol. 10(1), pages 167-179.
    2. Christoffersen, Peter, 2011. "Elements of Financial Risk Management," Elsevier Monographs, Elsevier, edition 2, number 9780123744487.
    3. Khandani, Amir E. & Kim, Adlar J. & Lo, Andrew W., 2010. "Consumer credit-risk models via machine-learning algorithms," Journal of Banking & Finance, Elsevier, vol. 34(11), pages 2767-2787, November.
    4. Edward I. Altman, 1968. "Financial Ratios, Discriminant Analysis And The Prediction Of Corporate Bankruptcy," Journal of Finance, American Finance Association, vol. 23(4), pages 589-609, September.
    5. Ohlson, Ja, 1980. "Financial Ratios And The Probabilistic Prediction Of Bankruptcy," Journal of Accounting Research, Wiley Blackwell, vol. 18(1), pages 109-131.
    6. West, Robert Craig, 1985. "A factor-analytic approach to bank condition," Journal of Banking & Finance, Elsevier, vol. 9(2), pages 253-266, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cakir, Murat, 2005. "Firma Başarısızlığının Dinamiklerinin Belirlenmesinde Makina Öğrenmesi Teknikleri: Ampirik Uygulamalar ve Karşılaştırmalı Analiz [Machine Learning Techniques in Determining the Dynamics of Corporat," MPRA Paper 55975, University Library of Munich, Germany.
    2. Zhou, Fanyin & Fu, Lijun & Li, Zhiyong & Xu, Jiawei, 2022. "The recurrence of financial distress: A survival analysis," International Journal of Forecasting, Elsevier, vol. 38(3), pages 1100-1115.
    3. Lin, Hsiou-Wei William & Lo, Huai-Chun & Wu, Ruei-Shian, 2016. "Modeling default prediction with earnings management," Pacific-Basin Finance Journal, Elsevier, vol. 40(PB), pages 306-322.
    4. Enrico Supino & Nicola Piras, 2022. "Le performance dei modelli di credit scoring in contesti di forte instabilit? macroeconomica: il ruolo delle Reti Neurali Artificiali," MANAGEMENT CONTROL, FrancoAngeli Editore, vol. 2022(2), pages 41-61.
    5. Lin, Fengyi & Yeh, Ching Chiang & Lee, Meng Yuan, 2013. "A Hybrid Business Failure Prediction Model Using Locally Linear Embedding And Support Vector Machines," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(1), pages 82-97, March.
    6. Lillian Cheung & Amnon Levy, 1998. "An integrative analysis of business bankruptcy in Australia," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 22(2), pages 149-167, June.
    7. Chen, An-Sing & Chu, Hsiang-Hui & Hung, Pi-Hsia & Cheng, Miao-Sih, 2020. "Financial risk and acquirers' stockholder wealth in mergers and acquisitions," The North American Journal of Economics and Finance, Elsevier, vol. 54(C).
    8. Taieb Hamadi & Sami El Omari, & Wafa Khlif, 2012. "Poids De L'Avis De L'Expert Comptable Judiciaire Dans La Decision Du Juge En Matiere De Redressement Judiciaire : Cas De La Tunisie," Post-Print hal-00937922, HAL.
    9. Youssef Zizi & Mohamed Oudgou & Abdeslam El Moudden, 2020. "Determinants and Predictors of SMEs’ Financial Failure: A Logistic Regression Approach," Risks, MDPI, vol. 8(4), pages 1-21, October.
    10. Carlos Serrano-Cinca & Yolanda Fuertes-Call鮠 & Bego uti鲲ez-Nieto & Beatriz Cuellar-Fernᮤez, 2014. "Path modelling to bankruptcy: causes and symptoms of the banking crisis," Applied Economics, Taylor & Francis Journals, vol. 46(31), pages 3798-3811, November.
    11. Nijskens, Rob & Mokas, Dimitris, 2019. "Credit Risk in Commercial Real Estate Bank Loans : The Role of Idiosyncratic versus Macro-Economic Factors," Other publications TiSEM ea4f2f0e-dc50-4987-91d3-6, Tilburg University, School of Economics and Management.
    12. Catherine Refait, 2004. "La prévision de la faillite fondée sur l’analyse financière de l’entreprise : un état des lieux," Économie et Prévision, Programme National Persée, vol. 162(1), pages 129-147.
    13. Noora Alzayed & Rasol Eskandari & Hassan Yazdifar, 2023. "Bank failure prediction: corporate governance and financial indicators," Review of Quantitative Finance and Accounting, Springer, vol. 61(2), pages 601-631, August.
    14. Balcaen, Sofie & Ooghe, Hubert, 2006. "35 years of studies on business failure: an overview of the classic statistical methodologies and their related problems," The British Accounting Review, Elsevier, vol. 38(1), pages 63-93.
    15. Nicoleta BARBUTA-MISU, 2011. "A Specific Model for Assessing the Financial Performance:Case study on Building Sector Enterprises of Galati County - Romania," Risk in Contemporary Economy, "Dunarea de Jos" University of Galati, Faculty of Economics and Business Administration, pages 318-325.
    16. Amin Jan & Maran Marimuthu & Muhammad Kashif Shad & Haseeb ur-Rehman & Muhammad Zahid & Ahmad Ali Jan, 2019. "Bankruptcy profile of the Islamic and conventional banks in Malaysia: a post-crisis period analysis," Economic Change and Restructuring, Springer, vol. 52(1), pages 67-87, February.
    17. repec:zbw:bofrdp:2009_035 is not listed on IDEAS
    18. Fayçal Mraihi, 2016. "Distressed Company Prediction Using Logistic Regression: Tunisian’s Case," Quarterly Journal of Business Studies, Research Academy of Social Sciences, vol. 2(1), pages 34-54.
    19. Hu, Yu-Chiang & Ansell, Jake, 2007. "Measuring retail company performance using credit scoring techniques," European Journal of Operational Research, Elsevier, vol. 183(3), pages 1595-1606, December.
    20. Demyanyk, Yuliya & Hasan, Iftekhar, 2009. "Financial crises and bank failures: a review of prediction methods," Bank of Finland Research Discussion Papers 35/2009, Bank of Finland.
    21. Kolari, James & Glennon, Dennis & Shin, Hwan & Caputo, Michele, 2002. "Predicting large US commercial bank failures," Journal of Economics and Business, Elsevier, vol. 54(4), pages 361-387.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jjrfmx:v:17:y:2024:i:2:p:50-:d:1328960. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.