IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1905.02810.html
   My bibliography  Save this paper

Decision Making with Machine Learning and ROC Curves

Author

Listed:
  • Kai Feng
  • Han Hong
  • Ke Tang
  • Jingyuan Wang

Abstract

The Receiver Operating Characteristic (ROC) curve is a representation of the statistical information discovered in binary classification problems and is a key concept in machine learning and data science. This paper studies the statistical properties of ROC curves and its implication on model selection. We analyze the implications of different models of incentive heterogeneity and information asymmetry on the relation between human decisions and the ROC curves. Our theoretical discussion is illustrated in the context of a large data set of pregnancy outcomes and doctor diagnosis from the Pre-Pregnancy Checkups of reproductive age couples in Henan Province provided by the Chinese Ministry of Health.

Suggested Citation

  • Kai Feng & Han Hong & Ke Tang & Jingyuan Wang, 2019. "Decision Making with Machine Learning and ROC Curves," Papers 1905.02810, arXiv.org.
    • Handle: RePEc:arx:papers:1905.02810
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1905.02810
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hong, Han & Preston, Bruce & Shum, Matthew, 2003. "Generalized Empirical Likelihood–Based Model Selection Criteria For Moment Condition Models," Econometric Theory, Cambridge University Press, vol. 19(6), pages 923-943, December.
    2. Janet Currie & W. Bentley MacLeod, 2017. "Diagnosing Expertise: Human Capital, Decision Making, and Performance among Physicians," Journal of Labor Economics, University of Chicago Press, vol. 35(1), pages 1-43.
    3. Hong, Han & Mahajan, Aprajit & Nekipelov, Denis, 2015. "Extremum estimation and numerical derivatives," Journal of Econometrics, Elsevier, vol. 188(1), pages 250-263.
    4. Andre Esteva & Brett Kuprel & Roberto A. Novoa & Justin Ko & Susan M. Swetter & Helen M. Blau & Sebastian Thrun, 2017. "Dermatologist-level classification of skin cancer with deep neural networks," Nature, Nature, vol. 542(7639), pages 115-118, February.
    5. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    6. Elliott, Graham & Lieli, Robert P., 2013. "Predicting binary outcomes," Journal of Econometrics, Elsevier, vol. 174(1), pages 15-26.
    7. Sherman, Robert P, 1993. "The Limiting Distribution of the Maximum Rank Correlation Estimator," Econometrica, Econometric Society, vol. 61(1), pages 123-137, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tarkocin, Coskun & Donduran, Murat, 2024. "Constructing early warning indicators for banks using machine learning models," The North American Journal of Economics and Finance, Elsevier, vol. 69(PB).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Susanne M. Schennach & Daniel Wilhelm, 2017. "A Simple Parametric Model Selection Test," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1663-1674, October.
    2. Shakeeb Khan & Fu Ouyang & Elie Tamer, 2020. "Inference on Semiparametric Multinomial Response Models," Discussion Papers Series 627, School of Economics, University of Queensland, Australia.
    3. Kai Feng & Han Hong & Ke Tang & Jingyuan Wang, 2023. "Statistical Tests for Replacing Human Decision Makers with Algorithms," Papers 2306.11689, arXiv.org, revised Dec 2024.
    4. Shakeeb Khan & Fu Ouyang & Elie Tamer, 2021. "Inference on semiparametric multinomial response models," Quantitative Economics, Econometric Society, vol. 12(3), pages 743-777, July.
    5. Hong, Han & Preston, Bruce, 2012. "Bayesian averaging, prediction and nonnested model selection," Journal of Econometrics, Elsevier, vol. 167(2), pages 358-369.
    6. Susanne M. Schennach, 2007. "Point estimation with exponentially tilted empirical likelihood," Papers 0708.1874, arXiv.org.
    7. Fu Ouyang & Thomas Tao Yang, 2020. "Semiparametric Discrete Choice Models for Bundles," Discussion Papers Series 625, School of Economics, University of Queensland, Australia.
    8. Fabrice Gilles & Sabina Issehnane & Florent Sari, 2022. "Using short-term jobs as a way to find a regular job. What kind of role for local context?," TEPP Working Paper 2022-07, TEPP.
    9. Paulo M. D. C. Parente & Richard J. Smith, 2021. "Quasi‐maximum likelihood and the kernel block bootstrap for nonlinear dynamic models," Journal of Time Series Analysis, Wiley Blackwell, vol. 42(4), pages 377-405, July.
    10. Cornelia Lawson, 2013. "Academic Inventions Outside the University: Investigating Patent Ownership in the UK," Industry and Innovation, Taylor & Francis Journals, vol. 20(5), pages 385-398, July.
    11. Patrick Bajari & Jeremy Fox & Stephen Ryan, 2008. "Evaluating wireless carrier consolidation using semiparametric demand estimation," Quantitative Marketing and Economics (QME), Springer, vol. 6(4), pages 299-338, December.
    12. Majd Oteibi & Adam Tamimi & Kaneez Abbas & Gabriel Tamimi & Danesh Khazaei & Hadi Khazaei, 2024. "Advancing Digital Health using AI and Machine Learning Solutions for Early Ultrasonic Detection of Breast Disorders in Women," International Journal of Research and Scientific Innovation, International Journal of Research and Scientific Innovation (IJRSI), vol. 11(11), pages 518-527, November.
    13. Vipin Arora & Shuping Shi, 2016. "Nonlinearities and tests of asset price bubbles," Empirical Economics, Springer, vol. 50(4), pages 1421-1433, June.
    14. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    15. Da Fonseca José & Grasselli Martino & Ielpo Florian, 2014. "Estimating the Wishart Affine Stochastic Correlation Model using the empirical characteristic function," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(3), pages 253-289, May.
    16. Hansen, Lars Peter & Heaton, John & Luttmer, Erzo G J, 1995. "Econometric Evaluation of Asset Pricing Models," The Review of Financial Studies, Society for Financial Studies, vol. 8(2), pages 237-274.
    17. Das, Marcel & van Soest, Arthur, 1999. "A panel data model for subjective information on household income growth," Journal of Economic Behavior & Organization, Elsevier, vol. 40(4), pages 409-426, December.
    18. Gillespie, Colin S., 2015. "Fitting Heavy Tailed Distributions: The poweRlaw Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 64(i02).
    19. Luis Garicano & Thomas N. Hubbard, 2016. "The Returns to Knowledge Hierarchies," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 32(4), pages 653-684.
    20. Yen, Steven T. & Chern, Wen S. & Lee, Hwang-Jaw, 1991. "Effects Of Income Sources On Household Food Expenditures," 1991 Annual Meeting, August 4-7, Manhattan, Kansas 271167, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1905.02810. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.