IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/29267.html
   My bibliography  Save this paper

Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System

Author

Listed:
  • Jens Ludwig
  • Sendhil Mullainathan

Abstract

Algorithms (in some form) are already widely used in the criminal justice system. We draw lessons from this experience for what is to come for the rest of society as machine learning diffuses. We find economists and other social scientists have a key role to play in shaping the impact of algorithms, in part through improving the tools used to build them.

Suggested Citation

  • Jens Ludwig & Sendhil Mullainathan, 2021. "Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System," NBER Working Papers 29267, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:29267
    Note: EH LE LS PE PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w29267.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    2. Kerwin Kofi Charles & Jonathan Guryan, 2011. "Studying Discrimination: Fundamental Challenges and Recent Progress," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 479-511, September.
    3. Richard Berk, 2019. "Accuracy and Fairness for Juvenile Justice Risk Assessments," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(1), pages 175-194, March.
    4. Anthony Heyes & Soodeh Saberian, 2019. "Temperature and Decisions: Evidence from 207,000 Court Cases," American Economic Journal: Applied Economics, American Economic Association, vol. 11(2), pages 238-265, April.
    5. Robert Bartlett & Adair Morse & Richard Stanton & Nancy Wallace, 2019. "Consumer-Lending Discrimination in the FinTech Era," NBER Working Papers 25943, National Bureau of Economic Research, Inc.
    6. Shamena Anwar & Hanming Fang, 2015. "Testing for Racial Prejudice in the Parole Board Release Process: Theory and Evidence," The Journal of Legal Studies, University of Chicago Press, vol. 44(1), pages 1-37.
    7. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    8. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    9. Daniel L. Chen & Tobias J. Moskowitz & Kelly Shue, 2016. "Decision Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1181-1242.
    10. Mark Hoekstra & CarlyWill Sloan, 2022. "Does Race Matter for Police Use of Force? Evidence from 911 Calls," American Economic Review, American Economic Association, vol. 112(3), pages 827-860, March.
    11. Stephen L. Ross & John Yinger, 2002. "The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262182289, April.
    12. Stéphane Mechoulan & Nicolas Sahuguet, 2015. "Assessing Racial Disparities in Parole Release," The Journal of Legal Studies, University of Chicago Press, vol. 44(1), pages 39-74.
    13. Ilyana Kuziemko, 2013. "How should inmates be released from prison? An assessment of parole versus fixed-sentence regimes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(1), pages 371-424.
    14. Chen, Daniel L. & Moskowitz, Tobias J. & Shue, Kelly, 2016. "Decision-Making Under the Gambler’s Fallacy: Evidence From Asylum Courts, Loan Officers, and Baseball Umpires," IAST Working Papers 16-43, Institute for Advanced Study in Toulouse (IAST).
    15. Richard A. Berk & Susan B. Sorenson & Geoffrey Barnes, 2016. "Forecasting Domestic Violence: A Machine Learning Approach to Help Inform Arraignment Decisions," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 13(1), pages 94-115, March.
    16. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    17. G. O. Mohler & M. B. Short & Sean Malinowski & Mark Johnson & G. E. Tita & Andrea L. Bertozzi & P. J. Brantingham, 2015. "Randomized Controlled Field Trials of Predictive Policing," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1399-1411, December.
    18. Roland G. Fryer, 2020. "An Empirical Analysis of Racial Differences in Police Use of Force: A Response," Journal of Political Economy, University of Chicago Press, vol. 128(10), pages 4003-4008.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Douglas Kiarelly Godoy de Araujo, 2023. "gingado: a machine learning library focused on economics and finance," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: applications and tools, volume 59, Bank for International Settlements.
    2. Runshan Fu & Ginger Zhe Jin & Meng Liu, 2022. "Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate," NBER Working Papers 29880, National Bureau of Economic Research, Inc.
    3. Gregory Weitzner, 2024. "Reputational Algorithm Aversion," Papers 2402.15418, arXiv.org, revised Jul 2024.
    4. Brendan O'Flaherty & Rajiv Sethi & Morgan Williams, 2024. "The nature, detection, and avoidance of harmful discrimination in criminal justice," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 43(1), pages 289-320, January.
    5. S. Mills & S. Costa & C. R. Sunstein, 2023. "AI, Behavioural Science, and Consumer Welfare," Journal of Consumer Policy, Springer, vol. 46(3), pages 387-400, September.
    6. Bjørkheim, Julie Brun & Nygård, Odd E., 2024. "Gender Differences in Tax Evasion: Evidence from Norwegian Administrative Data," Discussion Papers 2024/8, Norwegian School of Economics, Department of Business and Management Science.
    7. Brendon McConnell & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "How do Parole Boards Respond to Large, Societal Shocks? Evidence from the 9/11 Terrorist Attacks," Working Papers 2023-010, Human Capital and Economic Opportunity Working Group.
    8. Luca Grilli & Sergio Mariotti & Riccardo Marzano, 2024. "Artificial intelligence and shapeshifting capitalism," Journal of Evolutionary Economics, Springer, vol. 34(2), pages 303-318, April.
    9. Disa M. Hynsjö & Luca Perdoni, 2024. "Mapping Out Institutional Discrimination: The Economic Effects of Federal “Redlining”," CESifo Working Paper Series 11098, CESifo.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yulin Liu & Luyao Zhang, 2022. "Cryptocurrency Valuation: An Explainable AI Approach," Papers 2201.12893, arXiv.org, revised Jul 2023.
    2. Meier, Armando N. & Levav, Jonathan & Meier, Stephan, 2020. "Early Release and Recidivism," IZA Discussion Papers 13035, Institute of Labor Economics (IZA).
    3. Julia Godfrey & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "The Effect of Parole Board Racial Composition on Prisoner Outcomes," Working Papers 2023-011, Human Capital and Economic Opportunity Working Group.
    4. Jonas Radbruch & Amelie Schiprowski, 2020. "Interview Sequences and the Formation of Subjective Assessments," ECONtribute Discussion Papers Series 045, University of Bonn and University of Cologne, Germany.
    5. Eric Fesselmeyer, 2021. "The impact of temperature on labor quality: Umpire accuracy in Major League Baseball," Southern Economic Journal, John Wiley & Sons, vol. 88(2), pages 545-567, October.
    6. Andreas Fuster & Paul Goldsmith‐Pinkham & Tarun Ramadorai & Ansgar Walther, 2022. "Predictably Unequal? The Effects of Machine Learning on Credit Markets," Journal of Finance, American Finance Association, vol. 77(1), pages 5-47, February.
    7. LaVoice, Jessica & Vamossy, Domonkos F., 2024. "Racial disparities in debt collection," Journal of Banking & Finance, Elsevier, vol. 164(C).
    8. Brendon McConnell & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "How do Parole Boards Respond to Large, Societal Shocks? Evidence from the 9/11 Terrorist Attacks," Working Papers 2023-010, Human Capital and Economic Opportunity Working Group.
    9. Miao Liu, 2022. "Assessing Human Information Processing in Lending Decisions: A Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 607-651, May.
    10. Reio Tanji, 2022. "Pitch Call Discrimination in Major League Baseball: The Effect on the Observed Performance and the Salaries," Discussion Papers in Economics and Business 22-02, Osaka University, Graduate School of Economics.
    11. Guido de Blasio & Alessio D'Ignazio & Marco Letta, 2020. "Predicting Corruption Crimes with Machine Learning. A Study for the Italian Municipalities," Working Papers 16/20, Sapienza University of Rome, DISS.
    12. Ewens, Michael, 2022. "Race and Gender in Entrepreneurial Finance," SocArXiv djf8z, Center for Open Science.
    13. Chen, Daniel L. & Philippe, Arnaud, 2023. "Clash of norms judicial leniency on defendant birthdays," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 324-344.
    14. Anthony Edo & Nicolas Jacquemet & Constantine Yannelis, 2019. "Language skills and homophilous hiring discrimination: Evidence from gender and racially differentiated applications," Review of Economics of the Household, Springer, vol. 17(1), pages 349-376, March.
    15. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    16. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2017. "Econom\'etrie et Machine Learning," Papers 1708.06992, arXiv.org, revised Mar 2018.
    17. Crespo, Cristian, 2020. "Two become one: improving the targeting of conditional cash transfers with a predictive model of school dropout," LSE Research Online Documents on Economics 123139, London School of Economics and Political Science, LSE Library.
    18. Erik Heilmann & Janosch Henze & Heike Wetzel, 2021. "Machine learning in energy forecasts with an application to high frequency electricity consumption data," MAGKS Papers on Economics 202135, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    19. repec:wyi:journl:002164 is not listed on IDEAS
    20. Maria R. Ibanez & Michael W. Toffel, 2020. "How Scheduling Can Bias Quality Assessment: Evidence from Food-Safety Inspections," Management Science, INFORMS, vol. 66(6), pages 2396-2416, June.
    21. Manuel J. García Rodríguez & Vicente Rodríguez Montequín & Francisco Ortega Fernández & Joaquín M. Villanueva Balsera, 2019. "Public Procurement Announcements in Spain: Regulations, Data Analysis, and Award Price Estimator Using Machine Learning," Complexity, Hindawi, vol. 2019, pages 1-20, November.

    More about this item

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • H0 - Public Economics - - General
    • K0 - Law and Economics - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:29267. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.