IDEAS home Printed from https://ideas.repec.org/p/ajr/sodwps/2021-09.html
   My bibliography  Save this paper

Creating Powerful and Interpretable Models with Regression Networks

Author

Listed:
  • Lachlan O'Neill

    (Faculty of Information Technology, Monash University)

  • Simon D Angus

    (Dept. of Economics & SoDa Laboratories, Monash Business School, Monash University)

  • Satya Borgohain

    (SoDa Laboratories, Monash Business School, Monash University)

  • Nader Chmait

    (Faculty of Information Technology, Monash University)

  • David Dowe

    (Faculty of Information Technology, Monash University)

Abstract

As the discipline has evolved, research in machine learning has been focused more and more on creating more powerful neural networks, without regard for the interpretability of these networks. Such “black-box models†yield state-of-the-art results, but we cannot understand why they make a particular decision or prediction. Sometimes this is acceptable, but often it is not. We propose a novel architecture, Regression Networks, which combines the power of neural networks with the understandability of regression analysis. While some methods for combining these exist in the literature, our architecture generalizes these approaches by taking interactions into account, offering the power of a dense neural network without forsaking interpretability. We demonstrate that the models exceed the state-of-the-art performance of interpretable models on several benchmark datasets, matching the power of a dense neural network. Finally, we discuss how these techniques can be generalized to other neural architectures, such as convolutional and recurrent neural networks.

Suggested Citation

  • Lachlan O'Neill & Simon D Angus & Satya Borgohain & Nader Chmait & David Dowe, 2021. "Creating Powerful and Interpretable Models with Regression Networks," SoDa Laboratories Working Paper Series 2021-09, Monash University, SoDa Laboratories.
  • Handle: RePEc:ajr:sodwps:2021-09
    as

    Download full text from publisher

    File URL: http://soda-wps.s3-website-ap-southeast-2.amazonaws.com/RePEc/ajr/sodwps/2021-09.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Harrison, David Jr. & Rubinfeld, Daniel L., 1978. "Hedonic housing prices and the demand for clean air," Journal of Environmental Economics and Management, Elsevier, vol. 5(1), pages 81-102, March.
    2. Kelley Pace, R. & Barry, Ronald, 1997. "Sparse spatial autoregressions," Statistics & Probability Letters, Elsevier, vol. 33(3), pages 291-297, May.
    3. Samuele Lo Piano, 2020. "Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-7, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Philippe Goulet Coulombe, 2022. "A Neural Phillips Curve and a Deep Output Gap," Working Papers 22-01, Chair in macroeconomics and forecasting, University of Quebec in Montreal's School of Management.
    2. Philippe Goulet Coulombe, 2022. "A Neural Phillips Curve and a Deep Output Gap," Papers 2202.04146, arXiv.org, revised Oct 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Palmquist, Raymond B., 2006. "Property Value Models," Handbook of Environmental Economics, in: K. G. Mäler & J. R. Vincent (ed.), Handbook of Environmental Economics, edition 1, volume 2, chapter 16, pages 763-819, Elsevier.
    2. Lucija Muehlenbachs & Elisheba Spiller & Christopher Timmins, 2015. "The Housing Market Impacts of Shale Gas Development," American Economic Review, American Economic Association, vol. 105(12), pages 3633-3659, December.
    3. Sandy Fréret & Denis Maguain, 2017. "The effects of agglomeration on tax competition: evidence from a two-regime spatial panel model on French data," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 24(6), pages 1100-1140, December.
    4. Jianhong Shi & Qian Yang & Xiongya Li & Weixing Song, 2017. "Effects of measurement error on a class of single-index varying coefficient regression models," Computational Statistics, Springer, vol. 32(3), pages 977-1001, September.
    5. Alexandre Xavier Ywata Carvalho & Pedro Henrique Melo Albuquerque & Gilberto Rezende de Almeida Junior & Rafael Dantas Guimarães & Camilo Rey Laureto, 2009. "Clusterização Hierárquica Espacial com Atributos Binários," Discussion Papers 1428, Instituto de Pesquisa Econômica Aplicada - IPEA.
    6. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    7. Villalonga, Belen, 2004. "Intangible resources, Tobin's q, and sustainability of performance differences," Journal of Economic Behavior & Organization, Elsevier, vol. 54(2), pages 205-230, June.
    8. Brockmeier, M., 1991. "Entwicklung und Aufhebung von Reinheitsgeboten im Nahrungsmittelbereich – Analyse und Bewertung," Proceedings “Schriften der Gesellschaft für Wirtschafts- und Sozialwissenschaften des Landbaues e.V.”, German Association of Agricultural Economists (GEWISOLA), vol. 27.
    9. Lavado, Rouselle F. & Barrios, Erniel B., 2010. "Spatial Stochastic Frontier Models," Discussion Papers DP 2010-08, Philippine Institute for Development Studies.
    10. Miles M Finney, 2017. "Air Quality and the Development of Los Angeles," The Review of Regional Studies, Southern Regional Science Association, vol. 47(3), pages 271-288, Fall.
    11. Terri Menke, 1987. "Economic Welfare and Urban Amenities Across Race-Sex Groups," Urban Studies, Urban Studies Journal Limited, vol. 24(2), pages 151-161, April.
    12. David Brasington & Don Haurin, 2005. "Capitalization of Parent, School, and Peer Group Components of School Quality into House Price," Departmental Working Papers 2005-04, Department of Economics, Louisiana State University.
    13. Suneel Babu Chatla, 2023. "Nonparametric inference for additive models estimated via simplified smooth backfitting," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 75(1), pages 71-97, February.
    14. Miller, Steve & Startz, Richard, 2019. "Feasible generalized least squares using support vector regression," Economics Letters, Elsevier, vol. 175(C), pages 28-31.
    15. Chunfang Zhao & Yingliang Wu & Yunfeng Chen & Guohua Chen, 2023. "Multiscale Effects of Hedonic Attributes on Airbnb Listing Prices Based on MGWR: A Case Study of Beijing, China," Sustainability, MDPI, vol. 15(2), pages 1-21, January.
    16. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    17. Prendergast, Luke A. & Li Wai Suen, Connie, 2011. "A new and practical influence measure for subsets of covariance matrix sample principal components with applications to high dimensional datasets," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 752-764, January.
    18. repec:asg:wpaper:1006 is not listed on IDEAS
    19. Tizheng Li & Xiaojuan Kang, 2022. "Variable selection of higher-order partially linear spatial autoregressive model with a diverging number of parameters," Statistical Papers, Springer, vol. 63(1), pages 243-285, February.
    20. Deac Dan Stelian & Schebesch Klaus Bruno, 2018. "Market Forecasts and Client Behavioral Data: Towards Finding Adequate Model Complexity," Studia Universitatis „Vasile Goldis” Arad – Economics Series, Sciendo, vol. 28(3), pages 50-75, September.
    21. James Hansen & James McDonald & Panayiotis Theodossiou & Brad Larsen, 2010. "Partially Adaptive Econometric Methods For Regression and Classification," Computational Economics, Springer;Society for Computational Economics, vol. 36(2), pages 153-169, August.

    More about this item

    Keywords

    machine learning; policy evaluation; neural networks; regression; classification;
    All these keywords.

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ajr:sodwps:2021-09. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ashani Amarasinghe The email address of this maintainer does not seem to be valid anymore. Please ask Ashani Amarasinghe to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.