IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v50y2021i1p3-44.html
   My bibliography  Save this article

Fairness in Criminal Justice Risk Assessments: The State of the Art

Author

Listed:
  • Richard Berk
  • Hoda Heidari
  • Shahin Jabbari
  • Michael Kearns
  • Aaron Roth

Abstract

Objectives: Discussions of fairness in criminal justice risk assessments typically lack conceptual precision. Rhetoric too often substitutes for careful analysis. In this article, we seek to clarify the trade-offs between different kinds of fairness and between fairness and accuracy. Methods: We draw on the existing literatures in criminology, computer science, and statistics to provide an integrated examination of fairness and accuracy in criminal justice risk assessments. We also provide an empirical illustration using data from arraignments. Results: We show that there are at least six kinds of fairness, some of which are incompatible with one another and with accuracy. Conclusions: Except in trivial cases, it is impossible to maximize accuracy and fairness at the same time and impossible simultaneously to satisfy all kinds of fairness. In practice, a major complication is different base rates across different legally protected groups. There is a need to consider challenging trade-offs. These lessons apply to applications well beyond criminology where assessments of risk can be used by decision makers. Examples include mortgage lending, employment, college admissions, child welfare, and medical diagnoses.

Suggested Citation

  • Richard Berk & Hoda Heidari & Shahin Jabbari & Michael Kearns & Aaron Roth, 2021. "Fairness in Criminal Justice Risk Assessments: The State of the Art," Sociological Methods & Research, , vol. 50(1), pages 3-44, February.
    • Handle: RePEc:sae:somere:v:50:y:2021:i:1:p:3-44
    DOI: 10.1177/0049124118782533
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124118782533
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124118782533?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dalvi-Esfahani, Mohammad & Mosharaf-Dehkordi, Mehdi & Leong, Lam Wai & Ramayah, T. & Jamal Kanaan-Jebna, Abdulkarim M., 2023. "Exploring the drivers of XAI-enhanced clinical decision support systems adoption: Insights from a stimulus-organism-response perspective," Technological Forecasting and Social Change, Elsevier, vol. 195(C).
    2. Nicholas Tenev, 2024. "De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data," Papers 2405.00910, arXiv.org.
    3. Emanuele Aliverti & Kristian Lum & James E. Johndrow & David B. Dunson, 2021. "Removing the influence of group variables in high‐dimensional predictive modelling," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 791-811, July.
    4. Christophe Hurlin & Christophe Perignon & Sébastien Saurin, 2021. "The Fairness of Credit Scoring Models," Working Papers hal-03501452, HAL.
    5. Kigerl, Alex & Hamilton, Zachary & Kowalski, Melissa & Mei, Xiaohan, 2022. "The great methods bake-off: Comparing performance of machine learning algorithms," Journal of Criminal Justice, Elsevier, vol. 82(C).
    6. Anna Langenberg & Shih-Chi Ma & Tatiana Ermakova & Benjamin Fabian, 2023. "Formal Group Fairness and Accuracy in Automated Decision Making," Mathematics, MDPI, vol. 11(8), pages 1-25, April.
    7. Friedle, Cosima, 2022. "How Can Fairness Tools Impact the Understanding of Fairness and the Processes Within a Machine Learning Development Team?," Junior Management Science (JUMS), Junior Management Science e. V., vol. 7(5), pages 1289-1300.
    8. Youmi Suk & Kyung T. Han, 2024. "A Psychometric Framework for Evaluating Fairness in Algorithmic Decision Making: Differential Algorithmic Functioning," Journal of Educational and Behavioral Statistics, , vol. 49(2), pages 151-172, April.
    9. Yoan Hermstrüwer & Pascal Langenbach, 2022. "Fair Governance with Humans and Machines," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2022_04, Max Planck Institute for Research on Collective Goods, revised 01 Mar 2023.
    10. Arthur Charpentier, 2022. "Quantifying fairness and discrimination in predictive models," Papers 2212.09868, arXiv.org.
    11. Federico Fioravanti & Iyad Rahwan & Fernando Tohmé, 2022. "Properties of Aggregation Operators Relevant for Ethical Decision Making in Artificial Intelligence," Working Papers 177, Red Nacional de Investigadores en Economía (RedNIE).
    12. Bansak, Kirk & Paulson, Elisabeth, 2023. "Public Opinion on Fairness and Efficiency for Algorithmic and Human Decision-Makers," OSF Preprints pghmx, Center for Open Science.
    13. Kozodoi, Nikita & Jacob, Johannes & Lessmann, Stefan, 2022. "Fairness in credit scoring: Assessment, implementation and profit implications," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1083-1094.
    14. Federico Fioravanti & Iyad Rahwan & Fernando Abel Tohm'e, 2022. "Classes of Aggregation Rules for Ethical Decision Making in Automated Systems," Papers 2206.05160, arXiv.org, revised Jun 2023.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:50:y:2021:i:1:p:3-44. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.