IDEAS home Printed from https://ideas.repec.org/p/boc/fsug22/19.html
   My bibliography  Save this paper

GAM(L)A: An econometric model for interpretable machine learning

Author

Listed:
  • Sullivan Hué

    (Aix-Marseille Université, AMSE)

Abstract

Despite their high predictive performance, random forest and gradient boosting are often considered as black boxes or uninterpretable models, which has raised concerns from practitioners and regulators. As an alternative, I propose to use partial linear models that are inherently interpretable. Specifically, this presentation introduces GAM-lasso (GAMLA) and GAM-autometrics (GAMA), denoted as GAM(L)A in short. GAM(L)A combines parametric and non-parametric functions to accurately capture linearities and nonlinearities prevailing between dependent and explanatory variables and a variable-selection procedure to control for overfitting issues. Estimation relies on a two-step procedure building upon the double residual method. I illustrate the predictive performance and interpretability of GAM(L)A on a regression and a classification problem. The results show that GAM(L)A outperforms parametric models augmented by quadratic, cubic, and interaction effects. Moreover, the results also suggest that the performance of GAM(L)A is not significantly different from that of random forest and gradient boosting.

Suggested Citation

  • Sullivan Hué, 2022. "GAM(L)A: An econometric model for interpretable machine learning," French Stata Users' Group Meetings 2022 19, Stata Users Group.
    • Handle: RePEc:boc:fsug22:19
    as

    Download full text from publisher

    File URL: http://repec.org/frsug2022/France22_Hue.pdf
    File Function: presentation materials
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christophe Hurlin & Christophe Pérignon, 2019. "Machine learning et nouvelles sources de données pour le scoring de crédit," Revue d'économie financière, Association d'économie financière, vol. 0(3), pages 21-50.
    2. Dumitrescu, Elena & Hué, Sullivan & Hurlin, Christophe & Tokpavi, Sessi, 2022. "Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1178-1192.
    3. Castle, Jennifer L. & Hendry, David F., 2010. "A low-dimension portmanteau test for non-linearity," Journal of Econometrics, Elsevier, vol. 158(2), pages 231-245, October.
    4. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    5. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    6. Bramoullé, Yann & Boucher, Vincent, 2020. "Binary Outcomes and Linear Interactions," CEPR Discussion Papers 15505, C.E.P.R. Discussion Papers.
    7. Nikita Kozodoi & Johannes Jacob & Stefan Lessmann, 2021. "Fairness in Credit Scoring: Assessment, Implementation and Profit Implications," Papers 2103.01907, arXiv.org, revised Jun 2022.
    8. Jurgen A. Doornik & Henrik Hansen, 2008. "An Omnibus Test for Univariate and Multivariate Normality," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 927-939, December.
    9. Robinson, Peter M, 1988. "Root- N-Consistent Semiparametric Regression," Econometrica, Econometric Society, vol. 56(4), pages 931-954, July.
    10. Hendry, David F., 2000. "Econometrics: Alchemy or Science?: Essays in Econometric Methodology," OUP Catalogue, Oxford University Press, number 9780198293545.
    11. Gunnarsson, Björn Rafn & vanden Broucke, Seppe & Baesens, Bart & Óskarsdóttir, María & Lemahieu, Wilfried, 2021. "Deep learning for credit scoring: Do or don’t?," European Journal of Operational Research, Elsevier, vol. 295(1), pages 292-305.
    12. Bertrand Candelon & Elena-Ivona Dumitrescu & Christophe Hurlin, 2012. "How to Evaluate an Early-Warning System: Toward a Unified Statistical Framework for Assessing Financial Crises Forecasting Methods," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 60(1), pages 75-113, April.
    13. Bracke, Philippe & Datta, Anupam & Jung, Carsten & Sen, Shayak, 2019. "Machine learning explainability in finance: an application to default risk analysis," Bank of England working papers 816, Bank of England.
    14. Godfrey, Leslie G, 1978. "Testing for Higher Order Serial Correlation in Regression Equations When the Regressors Include Lagged Dependent Variables," Econometrica, Econometric Society, vol. 46(6), pages 1303-1310, November.
    15. David F. Hendry & Søren Johansen, 2011. "The Properties of Model Selection when Retaining Theory Variables," CREATES Research Papers 2011-36, Department of Economics and Business Economics, Aarhus University.
    16. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    17. Søren Johansen & Bent Nielsen, 2016. "Asymptotic Theory of Outlier Detection Algorithms for Linear Time Series Regression Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(2), pages 321-348, June.
    18. Desai, Vijay S. & Crook, Jonathan N. & Overstreet, George A., 1996. "A comparison of neural networks and linear scoring models in the credit union environment," European Journal of Operational Research, Elsevier, vol. 95(1), pages 24-37, November.
    19. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2018. "Econometrics and Machine Learning," Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 505-506, pages 147-169.
    20. Michael C. Lovell, 1963. "Seasonal Adjustment of Economic Time Series and Multiple Regression," Cowles Foundation Discussion Papers 151, Cowles Foundation for Research in Economics, Yale University.
    21. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    22. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    23. Finlay, Steven, 2011. "Multiple classifier architectures and their application to credit risk assessment," European Journal of Operational Research, Elsevier, vol. 210(2), pages 368-378, April.
    24. Lessmann, Stefan & Baesens, Bart & Seow, Hsin-Vonn & Thomas, Lyn C., 2015. "Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research," European Journal of Operational Research, Elsevier, vol. 247(1), pages 124-136.
    25. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2017. "Econom\'etrie et Machine Learning," Papers 1708.06992, arXiv.org, revised Mar 2018.
    26. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    27. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    28. B Baesens & T Van Gestel & S Viaene & M Stepanova & J Suykens & J Vanthienen, 2003. "Benchmarking state-of-the-art classification algorithms for credit scoring," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(6), pages 627-635, June.
    29. Daniel W. Apley & Jingyu Zhu, 2020. "Visualizing the effects of predictor variables in black box supervised learning models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(4), pages 1059-1086, September.
    30. Castle, Jennifer L. & Clements, Michael P. & Hendry, David F., 2013. "Forecasting by factors, by variables, by both or neither?," Journal of Econometrics, Elsevier, vol. 177(2), pages 305-319.
    31. Harrison, David Jr. & Rubinfeld, Daniel L., 1978. "Hedonic housing prices and the demand for clean air," Journal of Environmental Economics and Management, Elsevier, vol. 5(1), pages 81-102, March.
    32. Paleologo, Giuseppe & Elisseeff, André & Antonini, Gianluca, 2010. "Subagging for credit scoring models," European Journal of Operational Research, Elsevier, vol. 201(2), pages 490-499, March.
    33. Søren Johansen & Bent Nielsen, 2016. "Rejoinder: Asymptotic Theory of Outlier Detection Algorithms for Linear Time Series Regression Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(2), pages 374-381, June.
    34. Kozodoi, Nikita & Jacob, Johannes & Lessmann, Stefan, 2022. "Fairness in credit scoring: Assessment, implementation and profit implications," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1083-1094.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emmanuel Flachaire & Gilles Hacheme & Sullivan Hu'e & S'ebastien Laurent, 2022. "GAM(L)A: An econometric model for interpretable Machine Learning," Papers 2203.11691, arXiv.org.
    2. Emmanuel Flachaire & Sullivan Hué & Sébastien Laurent & Gilles Hacheme, 2024. "Interpretable Machine Learning Using Partial Linear Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 86(3), pages 519-540, June.
    3. Elena Ivona DUMITRESCU & Sullivan HUE & Christophe HURLIN & Sessi TOKPAVI, 2020. "Machine Learning or Econometrics for Credit Scoring: Let’s Get the Best of Both Worlds," LEO Working Papers / DR LEO 2839, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    4. Dumitrescu, Elena & Hué, Sullivan & Hurlin, Christophe & Tokpavi, Sessi, 2022. "Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1178-1192.
    5. David F. Hendry & Grayham E. Mizon, 2016. "Improving the teaching of econometrics," Cogent Economics & Finance, Taylor & Francis Journals, vol. 4(1), pages 1170096-117, December.
    6. Castle, Jennifer L. & Doornik, Jurgen A. & Hendry, David F., 2023. "Robust Discovery of Regression Models," Econometrics and Statistics, Elsevier, vol. 26(C), pages 31-51.
    7. Jurgen A. Doornik & David F. Hendry & Steve Cook, 2015. "Statistical model selection with “Big Data”," Cogent Economics & Finance, Taylor & Francis Journals, vol. 3(1), pages 1045216-104, December.
    8. Tigges, Maximilian & Mestwerdt, Sönke & Tschirner, Sebastian & Mauer, René, 2024. "Who gets the money? A qualitative analysis of fintech lending and credit scoring through the adoption of AI and alternative data," Technological Forecasting and Social Change, Elsevier, vol. 205(C).
    9. Castle, Jennifer L. & Doornik, Jurgen A. & Hendry, David F., 2021. "Modelling non-stationary ‘Big Data’," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1556-1575.
    10. David F. Hendry & Felix Pretis, 2013. "Anthropogenic influences on atmospheric CO2," Chapters, in: Roger Fouquet (ed.), Handbook on Energy and Climate Change, chapter 12, pages 287-326, Edward Elgar Publishing.
    11. Jennifer Castle & David Hendry, 2013. "Semi-automatic Non-linear Model selection," Economics Series Working Papers 654, University of Oxford, Department of Economics.
    12. Chen, Yujia & Calabrese, Raffaella & Martin-Barragan, Belen, 2024. "Interpretable machine learning for imbalanced credit scoring datasets," European Journal of Operational Research, Elsevier, vol. 312(1), pages 357-372.
    13. Ericsson Neil R., 2016. "Testing for and estimating structural breaks and other nonlinearities in a dynamic monetary sector," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 20(4), pages 377-398, September.
    14. Dangxing Chen & Weicheng Ye & Jiahui Ye, 2022. "Interpretable Selective Learning in Credit Risk," Papers 2209.10127, arXiv.org.
    15. Andrew B. Martinez, 2020. "Forecast Accuracy Matters for Hurricane Damage," Econometrics, MDPI, vol. 8(2), pages 1-24, May.
    16. Ericsson, Neil R., 2016. "Eliciting GDP forecasts from the FOMC’s minutes around the financial crisis," International Journal of Forecasting, Elsevier, vol. 32(2), pages 571-583.
    17. Stillwagon, Josh R., 2016. "Non-linear exchange rate relationships: An automated model selection approach with indicator saturation," The North American Journal of Economics and Finance, Elsevier, vol. 37(C), pages 84-109.
    18. Chen, Dangxing & Ye, Jiahui & Ye, Weicheng, 2023. "Interpretable selective learning in credit risk," Research in International Business and Finance, Elsevier, vol. 65(C).
    19. David F. Hendry, 2020. "First in, First out: Econometric Modelling of UK Annual CO_2 Emissions, 1860–2017," Economics Papers 2020-W02, Economics Group, Nuffield College, University of Oxford.
    20. Jennifer L. Castle & Jurgen A. Doornik & David F. Hendry & Ragnar Nymoen, 2014. "Misspecification Testing: Non-Invariance of Expectations Models of Inflation," Econometric Reviews, Taylor & Francis Journals, vol. 33(5-6), pages 553-574, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boc:fsug22:19. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F Baum (email available below). General contact details of provider: https://edirc.repec.org/data/stataea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.