IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2111.07889.html
   My bibliography  Save this paper

An Outcome Test of Discrimination for Ranked Lists

Author

Listed:
  • Jonathan Roth
  • Guillaume Saint-Jacques
  • YinYin Yu

Abstract

This paper extends Becker (1957)'s outcome test of discrimination to settings where a (human or algorithmic) decision-maker produces a ranked list of candidates. Ranked lists are particularly relevant in the context of online platforms that produce search results or feeds, and also arise when human decisionmakers express ordinal preferences over a list of candidates. We show that non-discrimination implies a system of moment inequalities, which intuitively impose that one cannot permute the position of a lower-ranked candidate from one group with a higher-ranked candidate from a second group and systematically improve the objective. Moreover, we show that that these moment inequalities are the only testable implications of non-discrimination when the auditor observes only outcomes and group membership by rank. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.

Suggested Citation

  • Jonathan Roth & Guillaume Saint-Jacques & YinYin Yu, 2021. "An Outcome Test of Discrimination for Ranked Lists," Papers 2111.07889, arXiv.org.
    • Handle: RePEc:arx:papers:2111.07889
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2111.07889
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kevin Lang & Ariella Kahn-Lang Spitzer, 2020. "Race Discrimination: An Economic Perspective," Journal of Economic Perspectives, American Economic Association, vol. 34(2), pages 68-89, Spring.
    2. Roth, Alvin E, 1984. "The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 991-1016, December.
    3. Molinari, Francesca, 2020. "Microeconometrics with partial identification," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 355-486, Elsevier.
    4. Molinari, Francesca, 2020. "Microeconometrics with partial identification," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 355-486, Elsevier.
    5. Castillo, Marco & Petrie, Ragan, 2010. "Discrimination in the lab: Does information trump appearance?," Games and Economic Behavior, Elsevier, vol. 68(1), pages 50-59, January.
    6. Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, January.
    7. Ashesh Rambachan & Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan, 2020. "An Economic Perspective on Algorithmic Fairness," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 91-95, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. YinYin Yu & Guillaume Saint-Jacques, 2022. "Choosing an algorithmic fairness metric for an online marketplace: Detecting and quantifying algorithmic bias on LinkedIn," Papers 2202.07300, arXiv.org, revised Aug 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuowen Chen & Hiroaki Kaido, 2022. "Robust Tests of Model Incompleteness in the Presence of Nuisance Parameters," Papers 2208.11281, arXiv.org, revised Sep 2023.
    2. Bei, Xinyue, 2024. "Local linearization based subvector inference in moment inequality models," Journal of Econometrics, Elsevier, vol. 238(1).
    3. Heiler, Phillip, 2024. "Heterogeneous treatment effect bounds under sample selection with an application to the effects of social media on political polarization," Journal of Econometrics, Elsevier, vol. 244(1).
    4. Levon Barseghyan & Maura Coughlin & Francesca Molinari & Joshua C. Teitelbaum, 2021. "Heterogeneous Choice Sets and Preferences," Econometrica, Econometric Society, vol. 89(5), pages 2015-2048, September.
    5. Jorg Stoye, 2020. "A Simple, Short, but Never-Empty Confidence Interval for Partially Identified Parameters," Papers 2010.10484, arXiv.org, revised Dec 2020.
    6. Kim, Dongwoo, 2023. "Partially identifying competing risks models: An application to the war on cancer," Journal of Econometrics, Elsevier, vol. 234(2), pages 536-564.
    7. Marcoux, Mathieu & Russell, Thomas M. & Wan, Yuanyuan, 2024. "A simple specification test for models with many conditional moment inequalities," Journal of Econometrics, Elsevier, vol. 242(1).
    8. Pietro Tebaldi & Alexander Torgovitsky & Hanbin Yang, 2023. "Nonparametric Estimates of Demand in the California Health Insurance Exchange," Econometrica, Econometric Society, vol. 91(1), pages 107-146, January.
    9. Aibo Gong, 2021. "Bounds for Treatment Effects in the Presence of Anticipatory Behavior," Papers 2111.06573, arXiv.org, revised Dec 2022.
    10. Jeff Dominitz & Charles F. Manski, 2024. "Using Total Margin of Error to Account for Non-Sampling Error in Election Polls: The Case of Nonresponse," Papers 2407.19339, arXiv.org, revised Oct 2024.
    11. Nathan Canen & Kristopher Ramsay, 2024. "Quantifying theory in politics: Identification, interpretation, and the role of structural methods," Journal of Theoretical Politics, , vol. 36(4), pages 301-327, October.
    12. Manski, Charles F. & Molinari, Francesca, 2021. "Estimating the COVID-19 infection rate: Anatomy of an inference problem," Journal of Econometrics, Elsevier, vol. 220(1), pages 181-192.
    13. Giustinelli, Pamela & Manski, Charles F. & Molinari, Francesca, 2022. "Tail and center rounding of probabilistic expectations in the Health and Retirement Study," Journal of Econometrics, Elsevier, vol. 231(1), pages 265-281.
    14. Xiaoyu Cheng, 2022. "Robust Data-Driven Decisions Under Model Uncertainty," Papers 2205.04573, arXiv.org.
    15. Chalak, Karim, 2024. "Nonparametric Gini-Frisch bounds," Journal of Econometrics, Elsevier, vol. 238(1).
    16. Wenlong Ji & Lihua Lei & Asher Spector, 2023. "Model-Agnostic Covariate-Assisted Inference on Partially Identified Causal Effects," Papers 2310.08115, arXiv.org, revised Nov 2024.
    17. Bryan S. Graham & Andrin Pelican, 2023. "Scenario sampling for large supermodular games," CeMMAP working papers 15/23, Institute for Fiscal Studies.
    18. Suguru Otani, 2021. "Estimating Endogenous Coalitional Mergers: Merger Costs and Assortativeness of Size and Specialization," Papers 2108.12744, arXiv.org, revised Mar 2023.
    19. Eric Gautier & Christiern Rose, 2022. "Fast, Robust Inference for Linear Instrumental Variables Models using Self-Normalized Moments," Papers 2211.02249, arXiv.org, revised Nov 2022.
    20. Vira Semenova, 2023. "Aggregated Intersection Bounds and Aggregated Minimax Values," Papers 2303.00982, arXiv.org, revised Jun 2024.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2111.07889. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.