IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/33104.html
   My bibliography  Save this paper

Unwarranted Disparity in High-Stakes Decisions: Race Measurement and Policy Responses

Author

Listed:
  • E. Jason Baron
  • Joseph J. Doyle Jr.
  • Natalia Emanuel
  • Peter Hull
  • Joseph P. Ryan

Abstract

Studies of racial discrimination often condition on endogenous measures of race or on earlier decisions that might themselves be affected by discrimination. We develop quasi-experimental tools for estimating the impact of racial misclassification on measures of unwarranted disparity, and for designing policy responses to unwarranted disparity that account for discrimination in earlier decisions. We apply these tools to the setting of child protective services (CPS), where previous work in our context has found that Black children are placed into foster care at higher rates than white children with identical potential for future maltreatment. CPS investigators misclassify 8–9% of Black and white children relative to their self-reported race, and this misclassification obscures around 24% of unwarranted disparity in foster care placement decisions. Policies that use algorithmic recommendations to eliminate total unwarranted disparity in placement rates are also meaningfully affected by earlier discrimination in CPS call screening.

Suggested Citation

  • E. Jason Baron & Joseph J. Doyle Jr. & Natalia Emanuel & Peter Hull & Joseph P. Ryan, 2024. "Unwarranted Disparity in High-Stakes Decisions: Race Measurement and Policy Responses," NBER Working Papers 33104, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:33104
    Note: CH LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w33104.pdf
    Download Restriction: Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    2. Max Gross & E. Jason Baron, 2022. "Temporary Stays and Persistent Gains: The Causal Effects of Foster Care," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 170-199, April.
    3. Amanda Y. Agan & Diag Davenport & Jens Ludwig & Sendhil Mullainathan, 2023. "Automating Automaticity: How the Context of Human Choice Affects the Extent of Algorithmic Bias," NBER Working Papers 30981, National Bureau of Economic Research, Inc.
    4. Peter Hull, 2021. "What Marginal Outcome Tests Can Tell Us About Racially Biased Decision-Making," NBER Working Papers 28503, National Bureau of Economic Research, Inc.
    5. Evan K. Rose, 2023. "A Constructivist Perspective on Empirical Discrimination Research," Journal of Economic Literature, American Economic Association, vol. 61(3), pages 906-923, September.
    6. Antle, Becky F. & Barbee, Anita P. & Christensen, Dana N. & Sullivan, Dana J., 2009. "The prevention of child maltreatment recidivism through the Solution-Based Casework model of child welfare practice," Children and Youth Services Review, Elsevier, vol. 31(12), pages 1346-1351, December.
    7. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    8. Benjamin Feigenberg & Conrad Miller, 2022. "Would Eliminating Racial Disparities in Motor Vehicle Searches have Efficiency Costs?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(1), pages 49-113.
    9. David Arnold & Will Dobbie & Peter Hull, 2021. "Measuring Racial Discrimination in Algorithms," AEA Papers and Proceedings, American Economic Association, vol. 111, pages 49-54, May.
    10. Ashesh Rambachan, 2022. "Identifying Prediction Mistakes in Observational Data," NBER Chapters, in: Economics of Artificial Intelligence, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yue-Yi Hwa & Clare Leaver, 2021. "Management in education systems," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 367-391.
    2. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    3. Hernández-Pizarro, Helena M. & Nicodemo, Catia & Casasnovas, Guillem López, 2020. "Discontinuous system of allowances: The response of prosocial health-care professionals," Journal of Public Economics, Elsevier, vol. 190(C).
    4. Ulrik Hvidman & Hans Henrik Sievertsen, 2021. "High-Stakes Grades and Student Behavior," Journal of Human Resources, University of Wisconsin Press, vol. 56(3), pages 821-849.
    5. Bertoni, Marco & Brunello, Giorgio & De Benedetto, Marco Alberto & De Paola, Maria, 2019. "External Monitors and Score Manipulation in Italian Schools: Symptomatic Treatment or Cure?," IZA Discussion Papers 12591, Institute of Labor Economics (IZA).
    6. Forbes, Silke J. & Lederman, Mara & Wither, Michael J., 2019. "Quality disclosure when firms set their own quality targets," International Journal of Industrial Organization, Elsevier, vol. 62(C), pages 228-250.
    7. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    8. Patrick Kline & Evan K Rose & Christopher R Walters, 2022. "Systemic Discrimination Among Large U.S. Employers [“Teachers and Student Achievement in the Chicago Public High Schools,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(4), pages 1963-2036.
    9. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    10. Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.
    11. Ashesh Rambachan, 2022. "Identifying Prediction Mistakes in Observational Data," NBER Chapters, in: Economics of Artificial Intelligence, National Bureau of Economic Research, Inc.
    12. Joshua Grossman & Julian Nyarko & Sharad Goel, 2023. "Racial bias as a multi‐stage, multi‐actor problem: An analysis of pretrial detention," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 20(1), pages 86-133, March.
    13. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    14. Berkhout, Emilie & Pradhan, Menno & Rahmawati, & Suryadarma, Daniel & Swarnata, Arya, 2024. "Using technology to prevent fraud in high stakes national school examinations: Evidence from Indonesia," Journal of Development Economics, Elsevier, vol. 170(C).
    15. Ferman, Bruno & Fontes, Luiz Felipe, 2022. "Assessing knowledge or classroom behavior? Evidence of teachers’ grading bias," Journal of Public Economics, Elsevier, vol. 216(C).
    16. Patrick Kline & Evan K Rose & Christopher R Walters, 2023. "Systemic Discrimination Among Large U.S. Employers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(4), pages 1963-2036.
    17. Brendan O'Flaherty & Rajiv Sethi & Morgan Williams, 2024. "The nature, detection, and avoidance of harmful discrimination in criminal justice," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 43(1), pages 289-320, January.
    18. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja, 2017. "Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets," American Economic Review, American Economic Association, vol. 107(6), pages 1535-1563, June.
    19. Piopiunik, Marc & Schwerdt, Guido & Woessmann, Ludger, 2013. "Central school exit exams and labor-market outcomes," European Journal of Political Economy, Elsevier, vol. 31(C), pages 93-108.
    20. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.

    More about this item

    JEL classification:

    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • I31 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - General Welfare, Well-Being
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth
    • J15 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Minorities, Races, Indigenous Peoples, and Immigrants; Non-labor Discrimination

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:33104. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.