IDEAS home Printed from https://ideas.repec.org/a/eee/teinso/v78y2024ics0160791x24002100.html
   My bibliography  Save this article

Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction

Author

Listed:
  • Hao, Xinyue
  • Demir, Emrah
  • Eyers, Daniel

Abstract

This paper explores the effects of integrating Generative Artificial Intelligence (GAI) into decision-making processes within organizations, employing a quasi-experimental pretest-posttest design. The study examines the synergistic interaction between Human Intelligence (HI) and GAI across four group decision-making scenarios within three global organizations renowned for their cutting-edge operational techniques. The research progresses through several phases: identifying research problems, collecting baseline data on decision-making, implementing AI interventions, and evaluating the outcomes post-intervention to identify shifts in performance. The results demonstrate that GAI effectively reduces human cognitive burdens and mitigates heuristic biases by offering data-driven support and predictive analytics, grounded in System 2 reasoning. This is particularly valuable in complex situations characterized by unfamiliarity and information overload, where intuitive, System 1 thinking is less effective. However, the study also uncovers challenges related to GAI integration, such as potential over-reliance on technology, intrinsic biases particularly ‘out-of-the-box’ thinking without contextual creativity. To address these issues, this paper proposes an innovative strategic framework for HI-GAI collaboration that emphasizes transparency, accountability, and inclusiveness.

Suggested Citation

  • Hao, Xinyue & Demir, Emrah & Eyers, Daniel, 2024. "Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction," Technology in Society, Elsevier, vol. 78(C).
    • Handle: RePEc:eee:teinso:v:78:y:2024:i:c:s0160791x24002100
    DOI: 10.1016/j.techsoc.2024.102662
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0160791X24002100
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techsoc.2024.102662?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Coeckelbergh, Mark & Sætra, Henrik Skaug, 2023. "Climate change and the political pathways of AI: The technocracy-democracy dilemma in light of artificial intelligence and human agency," Technology in Society, Elsevier, vol. 75(C).
    2. Jarrahi, Mohammad Hossein, 2018. "Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making," Business Horizons, Elsevier, vol. 61(4), pages 577-586.
    3. Duan, Yanqing & Edwards, John S. & Dwivedi, Yogesh K, 2019. "Artificial intelligence for decision making in the era of Big Data – evolution, challenges and research agenda," International Journal of Information Management, Elsevier, vol. 48(C), pages 63-71.
    4. Ostheimer, Julia & Chowdhury, Soumitra & Iqbal, Sarfraz, 2021. "An alliance of humans and machines for machine learning: Hybrid intelligent systems and their design principles," Technology in Society, Elsevier, vol. 66(C).
    5. Glöckner, Andreas & Betsch, Tilmann, 2008. "Modeling option and strategy choices with connectionist networks: Towards an integrative model of automatic and deliberate decision making," Judgment and Decision Making, Cambridge University Press, vol. 3(3), pages 215-228, March.
    6. Villa, Sebastián & Castañeda, Jaime Andrés, 2018. "Transshipments in supply chains: A behavioral investigation," European Journal of Operational Research, Elsevier, vol. 269(2), pages 715-729.
    7. Charles Ka Yui Leung & Joe Cho Yiu Ng & Edward Chi Ho TANG, 2019. "What do we know about Housing Supply? The case of Hong Kong," GRU Working Paper Series GRU_2019_012, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    8. Huang, Lei & Ladikas, Miltos & Schippl, Jens & He, Guangxi & Hahn, Julia, 2023. "Knowledge mapping of an artificial intelligence application scenario: A bibliometric analysis of the basic research of data-driven autonomous vehicles," Technology in Society, Elsevier, vol. 75(C).
    9. Yuhui Jing & Haoming Wang & Xiaojiao Chen & Chengliang Wang, 2024. "What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    10. Nima Kordzadeh & Maryam Ghasemaghaei, 2022. "Algorithmic bias: review, synthesis, and future research directions," European Journal of Information Systems, Taylor & Francis Journals, vol. 31(3), pages 388-409, May.
    11. Khogali, Hisham O. & Mekid, Samir, 2023. "The blended future of automation and AI: Examining some long-term societal and ethical impact features," Technology in Society, Elsevier, vol. 73(C).
    12. Sharif, Amir M. & Irani, Zahir, 2006. "Exploring Fuzzy Cognitive Mapping for IS Evaluation," European Journal of Operational Research, Elsevier, vol. 173(3), pages 1175-1187, September.
    13. Hangl, Johannes & Krause, Simon & Behrens, Viktoria Joy, 2023. "Drivers, barriers and social considerations for AI adoption in SCM," Technology in Society, Elsevier, vol. 74(C).
    14. Meissner, Philip & Wulf, Torsten, 2013. "Cognitive benefits of scenario planning: Its impact on biases and decision quality," Technological Forecasting and Social Change, Elsevier, vol. 80(4), pages 801-814.
    15. Christoph Werner & Anca M. Hanea & Oswaldo Morales-Nápoles, 2018. "Eliciting Multivariate Uncertainty from Experts: Considerations and Approaches Along the Expert Judgement Process," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 171-210, Springer.
    16. Bin-Nashwan, Saeed Awadh & Sadallah, Mouad & Bouteraa, Mohamed, 2023. "Use of ChatGPT in academia: Academic integrity hangs in the balance," Technology in Society, Elsevier, vol. 75(C).
    17. Scott, Rodney J & Cavana, Robert Y & Cameron, Donald, 2016. "Recent evidence on the effectiveness of group model building," European Journal of Operational Research, Elsevier, vol. 249(3), pages 908-918.
    18. Andreas Glöckner & Tilmann Betsch, 2008. "Modeling Option and Strategy Choices with Connectionist Networks: Towards an Integrative Model of Automatic and Deliberate Decision Making," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2008_02, Max Planck Institute for Research on Collective Goods.
    19. Sutton, Steve G. & Arnold, Vicky, 2013. "Focus group methods: Using interactive and nominal groups to explore emerging technology-driven phenomena in accounting and information systems," International Journal of Accounting Information Systems, Elsevier, vol. 14(2), pages 81-88.
    20. Krawinkler, Andreas & Breitenecker, Robert J. & Maresch, Daniela, 2022. "Heuristic decision-making in the green energy context:Bringing together simple rules and data-driven mathematical optimization," Technological Forecasting and Social Change, Elsevier, vol. 180(C).
    21. Monica D. Rosenberg & B. J. Casey & Avram J. Holmes, 2018. "Prediction complements explanation in understanding the developing brain," Nature Communications, Nature, vol. 9(1), pages 1-13, December.
    22. Samadhiya, Ashutosh & Yadav, Sanjeev & Kumar, Anil & Majumdar, Abhijit & Luthra, Sunil & Garza-Reyes, Jose Arturo & Upadhyay, Arvind, 2023. "The influence of artificial intelligence techniques on disruption management: Does supply chain dynamism matter?," Technology in Society, Elsevier, vol. 75(C).
    23. Haris Krijestorac & Rajiv Garg & Prabhudev Konana, 2021. "Decisions Under the Illusion of Objectivity: Digital Embeddedness and B2B Purchasing," Production and Operations Management, Production and Operations Management Society, vol. 30(7), pages 2232-2251, July.
    24. Li, Jing-Ping & Mirza, Nawazish & Rahat, Birjees & Xiong, Deping, 2020. "Machine learning and credit ratings prediction in the age of fourth industrial revolution," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    25. Berman, Alexander & de Fine Licht, Karl & Carlsson, Vanja, 2024. "Trustworthy AI in the public sector: An empirical analysis of a Swedish labor market decision-support system," Technology in Society, Elsevier, vol. 76(C).
    26. Giordano, Vito & Spada, Irene & Chiarello, Filippo & Fantoni, Gualtiero, 2024. "The impact of ChatGPT on human skills: A quantitative study on twitter data," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    27. Jackson, Thomas W. & Farzaneh, Pourya, 2012. "Theory-based model of factors affecting information overload," International Journal of Information Management, Elsevier, vol. 32(6), pages 523-532.
    28. Emrah Demir & Katy Huckle & Aris Syntetos & Andrew Lahy & Mike Wilson, 2019. "Vehicle Routing Problem: Past and Future," Springer Books, in: Peter Wells (ed.), Contemporary Operations and Logistics, chapter 0, pages 97-117, Springer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pietronudo, Maria Cristina & Croidieu, Grégoire & Schiavone, Francesco, 2022. "A solution looking for problems? A systematic literature review of the rationalizing influence of artificial intelligence on decision-making in innovation management," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    2. Rice, Stephen & Crouse, Sean R. & Winter, Scott R. & Rice, Connor, 2024. "The advantages and limitations of using ChatGPT to enhance technological research," Technology in Society, Elsevier, vol. 76(C).
    3. Volkmar, Gioia & Fischer, Peter M. & Reinecke, Sven, 2022. "Artificial Intelligence and Machine Learning: Exploring drivers, barriers, and future developments in marketing management," Journal of Business Research, Elsevier, vol. 149(C), pages 599-614.
    4. Armenia, Stefano & Franco, Eduardo & Iandolo, Francesca & Maielli, Giuliano & Vito, Pietro, 2024. "Zooming in and out the landscape: Artificial intelligence and system dynamics in business and management," Technological Forecasting and Social Change, Elsevier, vol. 200(C).
    5. Ida Merete Enholm & Emmanouil Papagiannidis & Patrick Mikalef & John Krogstie, 2022. "Artificial Intelligence and Business Value: a Literature Review," Information Systems Frontiers, Springer, vol. 24(5), pages 1709-1734, October.
    6. Carmen Isensee & Kai-Michael Griese & Frank Teuteberg, 2021. "Sustainable artificial intelligence: A corporate culture perspective [Sustainable artificial intelligence: Eine unternehmenskulturelle Perspektive]," Sustainability Nexus Forum, Springer, vol. 29(3), pages 217-230, December.
    7. Purva Grover & Arpan Kumar Kar & Yogesh K. Dwivedi, 2022. "Understanding artificial intelligence adoption in operations management: insights from the review of academic literature and social media discussions," Annals of Operations Research, Springer, vol. 308(1), pages 177-213, January.
    8. Shivam Gupta & Sachin Modgil & Samadrita Bhattacharyya & Indranil Bose, 2022. "Artificial intelligence for decision support systems in the field of operations research: review and future scope of research," Annals of Operations Research, Springer, vol. 308(1), pages 215-274, January.
    9. Manal Ahdadou & Abdellah Aajly & Mohamed Tahrouch, 2024. "Unlocking the potential of augmented intelligence: a discussion on its role in boardroom decision-making," International Journal of Disclosure and Governance, Palgrave Macmillan, vol. 21(3), pages 433-446, September.
    10. Sabhi Rajae & Abdelbaki Jamal Eddine & Taouab Omar & Eddaoudi Faissal & Abdelbaki Noureddine, 2024. "Managing emotions and algorithms: the delicate equilibrium between artificial intelligence and behavioral finance [Gérer les émotions et les algorithmes : l'équilibre délicat entre l'intelligence a," Post-Print hal-04644322, HAL.
    11. Zeng, Ying & Liu, Xinyi & Zhang, Xinyuan & Li, Zhiyong, 2024. "Retrospective of interdisciplinary research on robot services (1954–2023): From parasitism to symbiosis," Technology in Society, Elsevier, vol. 78(C).
    12. Monica Lestari Paramita & Maria Kasinidou & Styliani Kleanthous & Paolo Rosso & Tsvi Kuflik & Frank Hopfgartner, 2024. "Towards improving user awareness of search engine biases: A participatory design approach," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(5), pages 581-599, May.
    13. Robertson, Jeandri & Ferreira, Caitlin & Botha, Elsamari & Oosthuizen, Kim, 2024. "Game changers: A generative AI prompt protocol to enhance human-AI knowledge co-construction," Business Horizons, Elsevier, vol. 67(5), pages 499-510.
    14. Meissner, Philip & Brands, Christian & Wulf, Torsten, 2017. "Quantifiying blind spots and weak signals in executive judgment: A structured integration of expert judgment into the scenario development process," International Journal of Forecasting, Elsevier, vol. 33(1), pages 244-253.
    15. Andreas Glöckner, 2009. "Investigating intuitive and deliberate processes statistically: The multiple-measure maximum likelihood strategy classification method," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 4(3), pages 186-199, April.
    16. Cano-Marin, Enrique & Mora-Cantallops, Marçal & Sánchez-Alonso, Salvador, 2023. "Twitter as a predictive system: A systematic literature review," Journal of Business Research, Elsevier, vol. 157(C).
    17. Charles Ka Yui Leung & Joe Cho Yiu Ng & Edward Tang, 2020. "Why is the Hong Kong Housing Market Unaffordable? Some Stylized Facts and Estimations," Globalization Institute Working Papers 380, Federal Reserve Bank of Dallas.
    18. Shardul Sharad Phadnis, 2021. "Advancing scenario planning theory: A commentary on Fergnani and Chermack, 2021," Futures & Foresight Science, John Wiley & Sons, vol. 3(3-4), September.
    19. António Madureira & Nico Baken & Harry Bouwman, 2011. "Value of digital information networks: a holonic framework," Netnomics, Springer, vol. 12(1), pages 1-30, April.
    20. Siliang Tong & Nan Jia & Xueming Luo & Zheng Fang, 2021. "The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance," Strategic Management Journal, Wiley Blackwell, vol. 42(9), pages 1600-1631, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:teinso:v:78:y:2024:i:c:s0160791x24002100. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/technology-in-society .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.