IDEAS home Printed from https://ideas.repec.org/p/drm/wpaper/2016-10.html
   My bibliography  Save this paper

Reject inference in application scorecards: evidence from France

Author

Listed:
  • Ha-Thu Nguyen

Abstract

Credit scoring models are commonly developed using only accepted Known Good/Bad (G/B) applications, called KGB model, because we only know the performance of those accepted in the past. Obviously, the KGB model is not indicative of the entire through-the-door population, and reject inference precisely attempts to address the bias by assigning an inferred G/B status to rejected applications. In this paper, we discuss the pros and cons of various reject inference techniques, and pitfalls to avoid when using them. We consider a real dataset of a major French consumer finance bank to assess the effectiveness of the practice of using reject inference. To do that, we rely on the logistic regression framework to model probabilities to become good/bad, and then validate the model performance with and without sample selection bias correction. Our main results can be summarized as follows. First, we show that the best reject inference technique is not necessarily the most complicated one: reweighting and parceling provide more accurate and relevant results than fuzzy augmentation and Heckmans two-stage correction. Second, disregarding rejected applications significantly impacts the forecast accuracy of the scorecard. Third, as the sum of standard errors dramatically reduces when the sample size increases, reject inference turns out to produce an improved representation of the population. Finally, reject inference appears to be an effective way to reduce overfitting in model selection.

Suggested Citation

  • Ha-Thu Nguyen, 2016. "Reject inference in application scorecards: evidence from France," EconomiX Working Papers 2016-10, University of Paris Nanterre, EconomiX.
  • Handle: RePEc:drm:wpaper:2016-10
    as

    Download full text from publisher

    File URL: http://economix.fr/pdf/dt/2016/WP_EcoX_2016-10.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Patrick Puhani, 2000. "The Heckman Correction for Sample Selection and Its Critique," Journal of Economic Surveys, Wiley Blackwell, vol. 14(1), pages 53-68, February.
    2. Meng, Chun-Lo & Schmidt, Peter, 1985. "On the Cost of Partial Observability in the Bivariate Probit Model," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 26(1), pages 71-85, February.
    3. Evžen Kocenda & Martin Vojtek, 2011. "Default Predictors in Retail Credit Scoring: Evidence from Czech Banking Data," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 47(6), pages 80-98, November.
    4. Wu, I-Ding & Hand, David J., 2007. "Handling selection bias when choosing actions in retail credit applications," European Journal of Operational Research, Elsevier, vol. 183(3), pages 1560-1568, December.
    5. G G Chen & T Åstebro, 2012. "Bound and collapse Bayesian reject inference for credit scoring," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 63(10), pages 1374-1387, October.
    6. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    7. Banasik, John & Crook, Jonathan, 2007. "Reject inference, augmentation, and sample selection," European Journal of Operational Research, Elsevier, vol. 183(3), pages 1582-1594, December.
    8. J Banasik & J Crook & L Thomas, 2003. "Sample selection bias in credit scoring models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(8), pages 822-832, August.
    9. Kiefer, Nicholas M. & Larson, C. Erik, 2006. "Specification and Informational Issues in Credit Scoring," Working Papers 06-11, Cornell University, Center for Analytic Economics.
    10. Reichert, Alan K & Cho, Chien-Ching & Wagner, George M, 1983. "An Examination of the Conceptual Issues Involved in Developing Credit-scoring Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 1(2), pages 101-114, April.
    11. Thomas, Lyn C., 2000. "A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers," International Journal of Forecasting, Elsevier, vol. 16(2), pages 149-172.
    12. Boyes, William J. & Hoffman, Dennis L. & Low, Stuart A., 1989. "An econometric analysis of the bank credit scoring problem," Journal of Econometrics, Elsevier, vol. 40(1), pages 3-14, January.
    13. Greene, William, 1998. "Sample selection in credit-scoring models1," Japan and the World Economy, Elsevier, vol. 10(3), pages 299-316, July.
    14. Eisenbeis, Robert A, 1977. "Pitfalls in the Application of Discriminant Analysis in Business, Finance, and Economics," Journal of Finance, American Finance Association, vol. 32(3), pages 875-900, June.
    15. Crook, Jonathan & Banasik, John, 2004. "Does reject inference really improve the performance of application scoring models?," Journal of Banking & Finance, Elsevier, vol. 28(4), pages 857-874, April.
    16. Poirier, Dale J., 1980. "Partial observability in bivariate probit models," Journal of Econometrics, Elsevier, vol. 12(2), pages 209-217, February.
    17. Bücker, Michael & van Kampen, Maarten & Krämer, Walter, 2013. "Reject inference in consumer credit scoring with nonignorable missing data," Journal of Banking & Finance, Elsevier, vol. 37(3), pages 1040-1045.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Qiang Liu & Yingtao Luo & Shu Wu & Zhen Zhang & Xiangnan Yue & Hong Jin & Liang Wang, 2022. "RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring," Papers 2206.00568, arXiv.org.
    2. Rogelio A. Mancisidor & Michael Kampffmeyer & Kjersti Aas & Robert Jenssen, 2019. "Deep Generative Models for Reject Inference in Credit Scoring," Papers 1904.11376, arXiv.org, revised Sep 2021.
    3. Mengnan Song & Jiasong Wang & Suisui Su, 2022. "Towards a Better Microcredit Decision," Papers 2209.07574, arXiv.org.
    4. Adrien Ehrhardt & Christophe Biernacki & Vincent Vandewalle & Philippe Heinrich & S'ebastien Beben, 2019. "R\'eint\'egration des refus\'es en Credit Scoring," Papers 1903.10855, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ha Thu Nguyen, 2016. "Reject inference in application scorecards: evidence from France," Working Papers hal-04141601, HAL.
    2. Rogelio A. Mancisidor & Michael Kampffmeyer & Kjersti Aas & Robert Jenssen, 2019. "Deep Generative Models for Reject Inference in Credit Scoring," Papers 1904.11376, arXiv.org, revised Sep 2021.
    3. Y Kim & S Y Sohn, 2007. "Technology scoring model considering rejected applicants and effect of reject inference," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 58(10), pages 1341-1347, October.
    4. Hussein A. Abdou & John Pointon, 2011. "Credit Scoring, Statistical Techniques And Evaluation Criteria: A Review Of The Literature," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 18(2-3), pages 59-88, April.
    5. Thi Mai Luong, 2020. "Selection Effects of Lender and Borrower Choices on Risk Measurement, Management and Prudential Regulation," PhD Thesis, Finance Discipline Group, UTS Business School, University of Technology, Sydney, number 3-2020, January-A.
    6. Mengnan Song & Jiasong Wang & Suisui Su, 2022. "Towards a Better Microcredit Decision," Papers 2209.07574, arXiv.org.
    7. Zhiyong Li & Xinyi Hu & Ke Li & Fanyin Zhou & Feng Shen, 2020. "Inferring the outcomes of rejected loans: an application of semisupervised clustering," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 631-654, February.
    8. Crook, Jonathan N. & Edelman, David B. & Thomas, Lyn C., 2007. "Recent developments in consumer credit risk assessment," European Journal of Operational Research, Elsevier, vol. 183(3), pages 1447-1465, December.
    9. J Banasik & J Crook & L Thomas, 2003. "Sample selection bias in credit scoring models," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(8), pages 822-832, August.
    10. G Verstraeten & D Van den Poel, 2005. "The impact of sample bias on consumer credit scoring performance and profitability," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 56(8), pages 981-992, August.
    11. Monir El Annas & Badreddine Benyacoub & Mohamed Ouzineb, 2023. "Semi-supervised adapted HMMs for P2P credit scoring systems with reject inference," Computational Statistics, Springer, vol. 38(1), pages 149-169, March.
    12. Filiz Garip, 2012. "An Integrated Analysis of Migration and Remittances: Modeling Migration as a Mechanism for Selection," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 31(5), pages 637-663, October.
    13. Ananish Chaudhuri & Pushkar Maitra, 1997. "Determinants of Land Tenure Contracts; Theory and Evidence from Rural India," Departmental Working Papers 199710, Rutgers University, Department of Economics.
    14. Crone, Sven F. & Finlay, Steven, 2012. "Instance sampling in credit scoring: An empirical study of sample size and balancing," International Journal of Forecasting, Elsevier, vol. 28(1), pages 224-238.
    15. Ha-Thu Nguyen, 2015. "How is credit scoring used to predict default in China?," EconomiX Working Papers 2015-1, University of Paris Nanterre, EconomiX.
    16. J Banasik & J Crook, 2010. "Reject inference in survival analysis by augmentation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 61(3), pages 473-485, March.
    17. Li, Phillip, 2011. "Estimation of sample selection models with two selection mechanisms," Computational Statistics & Data Analysis, Elsevier, vol. 55(2), pages 1099-1108, February.
    18. Erwan Quintin & John J. Stevens, 2005. "Raising the bar for models of turnover," Finance and Economics Discussion Series 2005-23, Board of Governors of the Federal Reserve System (U.S.).
    19. Bücker, Michael & van Kampen, Maarten & Krämer, Walter, 2013. "Reject inference in consumer credit scoring with nonignorable missing data," Journal of Banking & Finance, Elsevier, vol. 37(3), pages 1040-1045.
    20. Maksym, Obrizan, 2010. "A Bayesian Model of Sample Selection with a Discrete Outcome Variable," MPRA Paper 28577, University Library of Munich, Germany.

    More about this item

    Keywords

    Reject inference; sample selection; selection bias; logistic regression; reweighting; parceling; fuzzy augmentation; Heckmans two-stage correction.;
    All these keywords.

    JEL classification:

    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:drm:wpaper:2016-10. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Valerie Mignon (email available below). General contact details of provider: https://edirc.repec.org/data/modemfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.