IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/8kvf4.html
   My bibliography  Save this paper

Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making

Author

Listed:
  • Veale, Michael
  • Van Kleek, Max
  • Binns, Reuben

Abstract

Cite as: Michael Veale, Max Van Kleek and Reuben Binns (2018) Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making. ACM Conference on Human Factors in Computing Systems (CHI'18). doi: 10.1145/3173574.3174014 Calls for heightened consideration of fairness and accountability in algorithmically-informed public decisions—like taxation, justice, and child protection—are now commonplace. How might designers support such human values? We interviewed 27 public sector machine learning practitioners across 5 OECD countries regarding challenges understanding and imbuing public values into their work. The results suggest a disconnect between organisational and institutional realities, constraints and needs, and those addressed by current research into usable, transparent and 'discrimination-aware' machine learning—absences likely to undermine practical initiatives unless addressed. We see design opportunities in this disconnect, such as in supporting the tracking of concept drift in secondary data sources, and in building usable transparency tools to identify risks and incorporate domain knowledge, aimed both at managers and at the `street-level bureaucrats' on the frontlines of public service. We conclude by outlining ethical challenges and future directions for collaboration in these high-stakes applications.

Suggested Citation

  • Veale, Michael & Van Kleek, Max & Binns, Reuben, 2018. "Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making," SocArXiv 8kvf4, Center for Open Science.
    • Handle: RePEc:osf:socarx:8kvf4
    DOI: 10.31219/osf.io/8kvf4
    as

    Download full text from publisher

    File URL: https://osf.io/download/5a774f0b68ee49000f397947/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/8kvf4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Aurélien Buffat, 2015. "Street-Level Bureaucracy and E-Government," Public Management Review, Taylor & Francis Journals, vol. 17(1), pages 149-161, January.
    2. Daniel Antony Kolkman & Paolo Campo & Tina Balke-Visser & Nigel Gilbert, 2016. "How to build models for government: criteria driving model acceptance in policymaking," Policy Sciences, Springer;Society of Policy Sciences, vol. 49(4), pages 489-504, December.
    3. Veale, Michael & Binns, Reuben, 2017. "Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data," SocArXiv ustxg, Center for Open Science.
    4. Edwards, Lilian & Veale, Michael, 2017. "Slave to the Algorithm? Why a 'right to an explanation' is probably not the remedy you are looking for," LawArXiv 97upg, Center for Open Science.
    5. Mary E. Thomson & Dilek Önkal & Ali Avcioğlu & Paul Goodwin, 2004. "Aviation Risk Perception: A Comparison Between Experts and Novices," Risk Analysis, John Wiley & Sons, vol. 24(6), pages 1585-1595, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kuziemski, Maciej & Misuraca, Gianluca, 2020. "AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings," Telecommunications Policy, Elsevier, vol. 44(6).
    2. Vesnic-Alujevic, Lucia & Nascimento, Susana & Pólvora, Alexandre, 2020. "Societal and ethical impacts of artificial intelligence: Critical notes on European policy frameworks," Telecommunications Policy, Elsevier, vol. 44(6).
    3. Veale, Michael & Brass, Irina, 2019. "Administration by Algorithm? Public Management meets Public Sector Machine Learning," SocArXiv mwhnb, Center for Open Science.
    4. Kathrin Hartmann & Georg Wenzelburger, 2021. "Uncertainty, risk and the use of algorithms in policy decisions: a case study on criminal justice in the USA," Policy Sciences, Springer;Society of Policy Sciences, vol. 54(2), pages 269-287, June.
    5. Matus, Kira & Veale, Michael, 2021. "Certification Systems for Machine Learning: Lessons from Sustainability," SocArXiv pm3wy, Center for Open Science.
    6. Fatima, Samar & Desouza, Kevin C. & Dawson, Gregory S., 2020. "National strategic artificial intelligence plans: A multi-dimensional analysis," Economic Analysis and Policy, Elsevier, vol. 67(C), pages 178-194.
    7. Kolkman, Daan, 2020. "The usefulness of algorithmic models in policy making," SocArXiv hpma8, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Veale, Michael, 2017. "Logics and practices of transparency and opacity in real-world applications of public sector machine learning," SocArXiv 6cdhe, Center for Open Science.
    2. Kira J.M. Matus & Michael Veale, 2022. "Certification systems for machine learning: Lessons from sustainability," Regulation & Governance, John Wiley & Sons, vol. 16(1), pages 177-196, January.
    3. Matus, Kira & Veale, Michael, 2021. "Certification Systems for Machine Learning: Lessons from Sustainability," SocArXiv pm3wy, Center for Open Science.
    4. Veale, Michael & Binns, Reuben & Van Kleek, Max, 2018. "Some HCI Priorities for GDPR-Compliant Machine Learning," LawArXiv wm6yk, Center for Open Science.
    5. König, Pascal D. & Wenzelburger, Georg, 2021. "The legitimacy gap of algorithmic decision-making in the public sector: Why it arises and how to address it," Technology in Society, Elsevier, vol. 67(C).
    6. Alina Köchling & Marius Claus Wehner, 2020. "Discriminated by an algorithm: a systematic review of discrimination and fairness by algorithmic decision-making in the context of HR recruitment and HR development," Business Research, Springer;German Academic Association for Business Research, vol. 13(3), pages 795-848, November.
    7. Salter, Mark B., 2007. "SeMS and sensibility: Security management systems and the management of risk in the Canadian Air Transport Security Authority," Journal of Air Transport Management, Elsevier, vol. 13(6), pages 389-398.
    8. Huiying Zhang & Zijian Zhu, 2024. "Mobile Government Service Promotion Strategies: Exploring Sustainable Development Pathways Based on Provincial Government Practices in China," Sustainability, MDPI, vol. 16(16), pages 1-21, August.
    9. Mitoko, Jeremiah, 2021. "Economics of Microcredit-From current crisis to new possibilities," MPRA Paper 108392, University Library of Munich, Germany.
    10. Hazel Si Min Lim & Araz Taeihagh, 2019. "Algorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities," Sustainability, MDPI, vol. 11(20), pages 1-28, October.
    11. Buhmann, Alexander & Fieseler, Christian, 2021. "Towards a deliberative framework for responsible innovation in artificial intelligence," Technology in Society, Elsevier, vol. 64(C).
    12. Cobbe, Jennifer & Veale, Michael & Singh, Jatinder, 2023. "Understanding Accountability in Algorithmic Supply Chains," SocArXiv p4sey, Center for Open Science.
    13. Kirsten Martin & Ari Waldman, 2023. "Are Algorithmic Decisions Legitimate? The Effect of Process and Outcomes on Perceptions of Legitimacy of AI Decisions," Journal of Business Ethics, Springer, vol. 183(3), pages 653-670, March.
    14. Vesnic-Alujevic, Lucia & Nascimento, Susana & Pólvora, Alexandre, 2020. "Societal and ethical impacts of artificial intelligence: Critical notes on European policy frameworks," Telecommunications Policy, Elsevier, vol. 44(6).
    15. Yi Hsin Lin & Yu Hern Chang, 2008. "Significant Factors of Aviation Insurance and Risk Management Strategy: An Empirical Study of Taiwanese Airline Carriers," Risk Analysis, John Wiley & Sons, vol. 28(2), pages 453-461, April.
    16. Söderlund, Kasia & Engström, Emma & Haresamudram, Kashyap & Larsson, Stefan & Strimling, Pontus, 2024. "Regulating high-reach AI: On transparency directions in the Digital Services Act," Internet Policy Review: Journal on Internet Regulation, Alexander von Humboldt Institute for Internet and Society (HIIG), Berlin, vol. 13(1), pages 1-31.
    17. Richard King & Kenan Okurut & Jo Herschan & Dan J. Lapworth & Rosalind Malcolm & Rory Moses McKeown & Katherine Pond, 2020. "Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda," Resources, MDPI, vol. 9(10), pages 1-17, October.
    18. Mazur Joanna, 2019. "Automated Decision-Making and the Precautionary Principle in EU Law," TalTech Journal of European Studies, Sciendo, vol. 9(4), pages 3-18, December.
    19. Daniela Sele & Marina Chugunova, 2023. "Putting a Human in the Loop: Increasing Uptake, but Decreasing Accuracy of Automated Decision-Making," Rationality and Competition Discussion Paper Series 438, CRC TRR 190 Rationality and Competition.
    20. Frederik Zuiderveen Borgesius & Joost Poort, 2017. "Online Price Discrimination and EU Data Privacy Law," Journal of Consumer Policy, Springer, vol. 40(3), pages 347-366, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:8kvf4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.