IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v11y2024i1d10.1057_s41599-024-03864-y.html
   My bibliography  Save this article

Inevitable challenges of autonomy: ethical concerns in personalized algorithmic decision-making

Author

Listed:
  • Wencheng Lu

    (Southeast University)

Abstract

Breakthroughs in Artificial Intelligence (AI) have transformed human decision-making processes. Individuals increasingly rely on algorithms to assist with various tasks such as searching and shopping. However, the widespread use of personalized algorithmic decision-making has raised numerous ethical concerns, specifically its impact on user autonomy. This article examines these concerns and argues that algorithmic decision-making presents several challenges to user autonomy that are difficult to eliminate. These challenges include the fact that algorithms deviate from a user’s authentic self, create self-reinforcing loops that narrow the user’s self, and lead to a decline in the user’s capacities. This article attributes these challenges to the ontological differences between users and AI, as well as their unawareness regarding the ethical risks associated with algorithms, asserting that the sense of autonomy users experience in algorithmic decision-making is merely an illusion. From the perspective of human-AI interaction, it is proposed a strategy that achieves a balance between human insight and AI through a human-in-the-loop approach to mitigate these challenges. This approach maximizes the strengths of AI in processing and analyzing data while ensuring that users retain agency.

Suggested Citation

  • Wencheng Lu, 2024. "Inevitable challenges of autonomy: ethical concerns in personalized algorithmic decision-making," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-9, December.
  • Handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-03864-y
    DOI: 10.1057/s41599-024-03864-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-024-03864-y
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1057/s41599-024-03864-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lena Bjørlo & Øystein Moen & Mark Pasquine, 2021. "The Role of Consumer Autonomy in Developing Sustainable AI: A Conceptual Framework," Sustainability, MDPI, vol. 13(4), pages 1-18, February.
    2. Leyer, Michael & Schneider, Sabrina, 2021. "Decision augmentation and automation with artificial intelligence: Threat or opportunity for managers?," Business Horizons, Elsevier, vol. 64(5), pages 711-724.
    3. Wien, Anders Hauge & Peluso, Alessandro M., 2021. "Influence of human versus AI recommenders: The roles of product type and cognitive processes," Journal of Business Research, Elsevier, vol. 137(C), pages 13-27.
    4. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    5. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    6. Susser, Daniel & Roessler, Beate & Nissenbaum, Helen, 2019. "Technology, autonomy, and manipulation," Internet Policy Review: Journal on Internet Regulation, Alexander von Humboldt Institute for Internet and Society (HIIG), Berlin, vol. 8(2), pages 1-22.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klockmann, Victor & von Schenk, Alicia & Villeval, Marie Claire, 2022. "Artificial intelligence, ethics, and intergenerational responsibility," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 284-317.
    2. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    3. Danielle Li & Lindsey R. Raymond & Peter Bergman, 2020. "Hiring as Exploration," NBER Working Papers 27736, National Bureau of Economic Research, Inc.
    4. Dargnies, Marie-Pierre & Hakimov, Rustamdjan & Kübler, Dorothea, 2022. "Aversion to hiring algorithms: Transparency, gender profiling, and self-confidence," Discussion Papers, Research Unit: Market Behavior SP II 2022-202, WZB Berlin Social Science Center.
    5. Armenia, Stefano & Franco, Eduardo & Iandolo, Francesca & Maielli, Giuliano & Vito, Pietro, 2024. "Zooming in and out the landscape: Artificial intelligence and system dynamics in business and management," Technological Forecasting and Social Change, Elsevier, vol. 200(C).
    6. Jean-Marie John-Mathews & Dominique Cardon & Christine Balagué, 2022. "From Reality to World. A Critical Perspective on AI Fairness," Journal of Business Ethics, Springer, vol. 178(4), pages 945-959, July.
    7. Lena V. Bjørlo, 2024. "Freedom from interference: Decisional privacy as a dimension of consumer privacy online," AMS Review, Springer;Academy of Marketing Science, vol. 14(1), pages 12-36, June.
    8. Bauer, Kevin & Gill, Andrej, 2021. "Mirror, mirror on the wall: Machine predictions and self-fulfilling prophecies," SAFE Working Paper Series 313, Leibniz Institute for Financial Research SAFE.
    9. Sarah Spiekermann & Hanna Krasnova & Oliver Hinz & Annika Baumann & Alexander Benlian & Henner Gimpel & Irina Heimbach & Antonia Köster & Alexander Maedche & Björn Niehaves & Marten Risius & Manuel Tr, 2022. "Values and Ethics in Information Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(2), pages 247-264, April.
    10. Runshan Fu & Ginger Zhe Jin & Meng Liu, 2022. "Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate," NBER Working Papers 29880, National Bureau of Economic Research, Inc.
    11. Fumagalli, Elena & Rezaei, Sarah & Salomons, Anna, 2022. "OK computer: Worker perceptions of algorithmic recruitment," Research Policy, Elsevier, vol. 51(2).
    12. Bauer, Kevin & Pfeuffer, Nicolas & Abdel-Karim, Benjamin M. & Hinz, Oliver & Kosfeld, Michael, 2020. "The terminator of social welfare? The economic consequences of algorithmic discrimination," SAFE Working Paper Series 287, Leibniz Institute for Financial Research SAFE.
    13. Maria De‐Arteaga & Stefan Feuerriegel & Maytal Saar‐Tsechansky, 2022. "Algorithmic fairness in business analytics: Directions for research and practice," Production and Operations Management, Production and Operations Management Society, vol. 31(10), pages 3749-3770, October.
    14. Jella Pfeiffer & Julia Gutschow & Christian Haas & Florian Möslein & Oliver Maspfuhl & Frederik Borgers & Suzana Alpsancar, 2023. "Algorithmic Fairness in AI," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 65(2), pages 209-222, April.
    15. Hermann, Erik & Puntoni, Stefano, 2024. "Artificial intelligence and consumer behavior: From predictive to generative AI," Journal of Business Research, Elsevier, vol. 180(C).
    16. Sophie-Charlotte Klose & Johannes Lederer, 2020. "A Pipeline for Variable Selection and False Discovery Rate Control With an Application in Labor Economics," Papers 2006.12296, arXiv.org, revised Jun 2020.
    17. Martinovici, A., 2019. "Revealing attention - how eye movements predict brand choice and moment of choice," Other publications TiSEM 7dca38a5-9f78-4aee-bd81-c, Tilburg University, School of Economics and Management.
    18. Aliprantis, Dionissi & Martin, Hal & Tauber, Kristen, 2024. "What determines the success of housing mobility programs?," Journal of Housing Economics, Elsevier, vol. 65(C).
    19. Wilson, Christopher & van der Velden, Maja, 2022. "Sustainable AI: An integrated model to guide public sector decision-making," Technology in Society, Elsevier, vol. 68(C).
    20. Daníelsson, Jón & Macrae, Robert & Uthemann, Andreas, 2022. "Artificial intelligence and systemic risk," Journal of Banking & Finance, Elsevier, vol. 140(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-03864-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.