IDEAS home Printed from https://ideas.repec.org/a/eee/eejocm/v33y2019ics1755534519300910.html
   My bibliography  Save this article

‘Computer says no’ is not enough: Using prototypical examples to diagnose artificial neural networks for discrete choice analysis

Author

Listed:
  • Alwosheel, Ahmad
  • van Cranenburgh, Sander
  • Chorus, Caspar G.

Abstract

Artificial Neural Networks (ANNs) are increasingly used for discrete choice analysis, being appreciated in particular for their strong predictive power. However, many choice modellers are critical – and rightfully so – about using ANNs, for the reason that they are hard to diagnose. That is, for analysts it is hard to see whether a trained (estimated) ANN has learned intuitively reasonable relationships, as opposed to spurious, inexplicable or otherwise undesirable ones. As a result, choice modellers often find it difficult to trust an ANN, even if its predictive performance is strong. Inspired by research from the field of computer vision, this paper pioneers a low-cost and easy-to-implement methodology to diagnose ANNs in the context of choice behaviour analysis. The method involves synthesising prototypical examples after having trained the ANN. These prototypical examples expose the fundamental relationships that the ANN has learned. These, in turn, can be evaluated by the analyst to see whether they make sense and are desirable, or not. In this paper we show how to use such prototypical examples in the context of choice data and we discuss practical considerations for successfully diagnosing ANNs. Furthermore, we cross-validate our findings using techniques from traditional discrete choice analysis. Our results suggest that the proposed method helps build trust in well-functioning ANNs, and is able to flag poorly trained ANNs. As such, it helps choice modellers use ANNs for choice behaviour analysis in a more reliable and effective way.

Suggested Citation

  • Alwosheel, Ahmad & van Cranenburgh, Sander & Chorus, Caspar G., 2019. "‘Computer says no’ is not enough: Using prototypical examples to diagnose artificial neural networks for discrete choice analysis," Journal of choice modelling, Elsevier, vol. 33(C).
  • Handle: RePEc:eee:eejocm:v:33:y:2019:i:c:s1755534519300910
    DOI: 10.1016/j.jocm.2019.100186
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1755534519300910
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jocm.2019.100186?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alwosheel, Ahmad & van Cranenburgh, Sander & Chorus, Caspar G., 2018. "Is your dataset big enough? Sample size requirements when using artificial neural networks for discrete choice analysis," Journal of choice modelling, Elsevier, vol. 28(C), pages 167-182.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ortelli, Nicola & Hillel, Tim & Pereira, Francisco C. & de Lapparent, Matthieu & Bierlaire, Michel, 2021. "Assisted specification of discrete choice models," Journal of choice modelling, Elsevier, vol. 39(C).
    2. Ali, Azam & Kalatian, Arash & Choudhury, Charisma F., 2023. "Comparing and contrasting choice model and machine learning techniques in the context of vehicle ownership decisions," Transportation Research Part A: Policy and Practice, Elsevier, vol. 173(C).
    3. S. Van Cranenburgh & S. Wang & A. Vij & F. Pereira & J. Walker, 2021. "Choice modelling in the age of machine learning -- discussion paper," Papers 2101.11948, arXiv.org, revised Nov 2021.
    4. Sander Cranenburgh & Marco Kouwenhoven, 2021. "An artificial neural network based method to uncover the value-of-travel-time distribution," Transportation, Springer, vol. 48(5), pages 2545-2583, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haoying Wang & Guohui Wu, 2022. "Modeling discrete choices with large fine-scale spatial data: opportunities and challenges," Journal of Geographical Systems, Springer, vol. 24(3), pages 325-351, July.
    2. Ester Vasta & Tommaso Scimone & Giovanni Nobile & Otto Eberhardt & Daniele Dugo & Massimiliano Maurizio De Benedetti & Luigi Lanuzza & Giuseppe Scarcella & Luca Patanè & Paolo Arena & Mario Cacciato, 2023. "Models for Battery Health Assessment: A Comparative Evaluation," Energies, MDPI, vol. 16(2), pages 1-34, January.
    3. Koffi Dumor & Li Yao, 2019. "Estimating China’s Trade with Its Partner Countries within the Belt and Road Initiative Using Neural Network Analysis," Sustainability, MDPI, vol. 11(5), pages 1-22, March.
    4. Mihai Mutascu & Scott W. Hegerty, 2023. "Predicting the contribution of artificial intelligence to unemployment rates: an artificial neural network approach," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 47(2), pages 400-416, June.
    5. Venkatraj, V. & Dixit, M.K., 2022. "Challenges in implementing data-driven approaches for building life cycle energy assessment: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 160(C).
    6. Mahmoud Abdel-Sattar & Adel M. Al-Saif & Abdulwahed M. Aboukarima & Dalia H. Eshra & Lidia Sas-Paszt, 2022. "Quality Attributes Prediction of Flame Seedless Grape Clusters Based on Nutritional Status Employing Multiple Linear Regression Technique," Agriculture, MDPI, vol. 12(9), pages 1-19, August.
    7. Bonfiglio, A. & Camaioni, B. & Carta, V. & Cristiano, S., 2023. "Estimating the common agricultural policy milestones and targets by neural networks," Evaluation and Program Planning, Elsevier, vol. 99(C).
    8. S. Van Cranenburgh & S. Wang & A. Vij & F. Pereira & J. Walker, 2021. "Choice modelling in the age of machine learning -- discussion paper," Papers 2101.11948, arXiv.org, revised Nov 2021.
    9. Ying Zhang & Mutahar Safdar & Jiarui Xie & Jinghao Li & Manuel Sage & Yaoyao Fiona Zhao, 2023. "A systematic review on data of additive manufacturing for machine learning applications: the data quality, type, preprocessing, and management," Journal of Intelligent Manufacturing, Springer, vol. 34(8), pages 3305-3340, December.
    10. Wang, Yuanping & Hou, Lingchun & Hu, Lang & Cai, Weiguang & Wang, Lin & Dai, Cuilian & Chen, Juntao, 2023. "How family structure type affects household energy consumption: A heterogeneous study based on Chinese household evidence," Energy, Elsevier, vol. 284(C).
    11. Sadia Samar Ali & Rajbir Kaur & D. Jinil Persis & Raiswa Saha & Murugan Pattusamy & V. Raja Sreedharan, 2023. "Developing a hybrid evaluation approach for the low carbon performance on sustainable manufacturing environment," Annals of Operations Research, Springer, vol. 324(1), pages 249-281, May.
    12. Kigerl, Alex & Hamilton, Zachary & Kowalski, Melissa & Mei, Xiaohan, 2022. "The great methods bake-off: Comparing performance of machine learning algorithms," Journal of Criminal Justice, Elsevier, vol. 82(C).
    13. Melvin Wong & Bilal Farooq, 2019. "Information processing constraints in travel behaviour modelling: A generative learning approach," Papers 1907.07036, arXiv.org, revised Jul 2019.
    14. Smeele, Nicholas V.R. & Chorus, Caspar G. & Schermer, Maartje H.N. & de Bekker-Grob, Esther W., 2023. "Towards machine learning for moral choice analysis in health economics: A literature review and research agenda," Social Science & Medicine, Elsevier, vol. 326(C).
    15. Sander Cranenburgh & Marco Kouwenhoven, 2021. "An artificial neural network based method to uncover the value-of-travel-time distribution," Transportation, Springer, vol. 48(5), pages 2545-2583, October.
    16. Horvath, Sabine & Soot, Matthias & Zaddach, Sebastian & Neuner, Hans & Weitkamp, Alexandra, 2021. "Deriving adequate sample sizes for ANN-based modelling of real estate valuation tasks by complexity analysis," Land Use Policy, Elsevier, vol. 107(C).
    17. Sophia Voulgaropoulou & Nikolaos Samaras & Nikolaos Ploskas, 2022. "Predicting the Execution Time of the Primal and Dual Simplex Algorithms Using Artificial Neural Networks," Mathematics, MDPI, vol. 10(7), pages 1-21, March.
    18. Melvin Wong & Bilal Farooq, 2019. "ResLogit: A residual neural network logit model for data-driven choice modelling," Papers 1912.10058, arXiv.org, revised Feb 2021.
    19. Salah Alghamdi & Waiching Tang & Sittimont Kanjanabootra & Dariusz Alterman, 2024. "Optimising Building Energy and Comfort Predictions with Intelligent Computational Model," Sustainability, MDPI, vol. 16(8), pages 1-19, April.
    20. Guillaume Coqueret & Romain Deguest, 2024. "Unexpected opportunities in misspecified predictive regressions," Post-Print hal-04595355, HAL.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eejocm:v:33:y:2019:i:c:s1755534519300910. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/journal-of-choice-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.