IDEAS home Printed from https://ideas.repec.org/p/dpr/wpaper/1233r.html
   My bibliography  Save this paper

Do people rely on ChatGPT more than their peers to detect deepfake news?

Author

Listed:
  • Yuhao Fu
  • Nobuyuki Hanaki

Abstract

This experimental study investigates whether people rely more on ChatGPT (GPT-4) than on their human peers when detecting AI-generated fake news (deepfake news). In multiple rounds of deepfake detection tasks conducted in a laboratory setting, student participants exhibited a greater reliance on ChatGPT compared to their peers. We explored this over-reliance on AI from two perspectives: the weight of advice (WOA) and the decomposition of reliance (DOR) into two stages. Our analysis indicates that reliance on external advice is primarily influenced by the source and quality of the advice, as well as the subjects’ prior beliefs, knowledge, and experience, while the type of news and time spent on tasks have no effect. Additionally, our study indicates a potential sequential mechanism of advice utilization, wherein the advice source affects reliance in both stages—activation and integration—whereas the quality of the advice, along with knowledge and experience, influences only the second stage. Our findings suggest that relying on AI to detect AI may not be detrimental and could, in fact, contribute to a deeper understanding of human-AI interaction and support advancements in AI development during the Generative Artificial Intelligence (GAI) era.

Suggested Citation

  • Yuhao Fu & Nobuyuki Hanaki, 2024. "Do people rely on ChatGPT more than their peers to detect deepfake news?," ISER Discussion Paper 1233r, Institute of Social and Economic Research, Osaka University, revised Dec 2024.
    • Handle: RePEc:dpr:wpaper:1233r
    as

    Download full text from publisher

    File URL: https://www.iser.osaka-u.ac.jp/library/dp/2024/DP1233R.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    2. Christopher Whyte, 2020. "Deepfake news: AI-enabled disinformation as a multi-level public policy challenge," Journal of Cyber Policy, Taylor & Francis Journals, vol. 5(2), pages 199-217, May.
    3. Mesbah, Neda & Tauchert, Christoph & Buxmann, Peter, 2021. "Whose Advice Counts More – Man or Machine? An Experimental Investigation of AI-based Advice Utilization," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 124796, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    4. Maggioni, Mario A. & Rossignoli, Domenico, 2023. "If it looks like a human and speaks like a human ... Communication and cooperation in strategic Human–Robot interactions," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 104(C).
    5. Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    6. Margarita Leib & Nils Köbis & Rainer Michael Rilke & Marloes Hagens & Bernd Irlenbusch, 2024. "Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty," The Economic Journal, Royal Economic Society, vol. 134(658), pages 766-784.
    7. Tse, Tiffany Tsz Kwan & Hanaki, Nobuyuki & Mao, Bolin, 2024. "Beware the performance of an algorithm before relying on it: Evidence from a stock price forecasting experiment," Journal of Economic Psychology, Elsevier, vol. 102(C).
    8. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:dpr:wpaper:1233 is not listed on IDEAS
    2. Tse, Tiffany Tsz Kwan & Hanaki, Nobuyuki & Mao, Bolin, 2024. "Beware the performance of an algorithm before relying on it: Evidence from a stock price forecasting experiment," Journal of Economic Psychology, Elsevier, vol. 102(C).
    3. Greiner, Ben & Grünwald, Philipp & Lindner, Thomas & Lintner, Georg & Wiernsperger, Martin, 2024. "Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study," Department for Strategy and Innovation Working Paper Series 01/2024, WU Vienna University of Economics and Business.
    4. Back, Camila & Morana, Stefan & Spann, Martin, 2023. "When do robo-advisors make us better investors? The impact of social design elements on investor behavior," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 103(C).
    5. Ivanova-Stenzel, Radosveta & Tolksdorf, Michel, 2023. "Measuring Preferences for Algorithms - Are people really algorithm averse after seeing the algorithm perform?," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277692, Verein für Socialpolitik / German Economic Association.
    6. Wendelin Schnedler & Nina Lucia Stephan, 2020. "Revisiting a Remedy Against Chains of Unkindness," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 72(3), pages 347-364, July.
    7. Kyung Hwan Baik & Subhasish M. Chowdhury & Abhijit Ramalingam, 2021. "Group size and matching protocol in contests," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 54(4), pages 1716-1736, November.
    8. David J. Cooper & Krista Saral & Marie Claire Villeval, 2021. "Why Join a Team?," Management Science, INFORMS, vol. 67(11), pages 6980-6997, November.
    9. Zakaria Babutsidze & Nobuyuki Hanaki & Adam Zylbersztejn, 2019. "Digital Communication and Swift Trust," Post-Print halshs-02409314, HAL.
    10. Galliera, Arianna, 2018. "Self-selecting random or cumulative pay? A bargaining experiment," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 72(C), pages 106-120.
    11. Julien Jacob & Eve-Angéline Lambert & Mathieu Lefebvre & Sarah Driessche, 2023. "Information disclosure under liability: an experiment on public bads," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 61(1), pages 155-197, July.
    12. Simon Gaechter & Chris Starmer & Fabio Tufano, 2022. "Measuring “group cohesion” to reveal the power of social relationships in team production," Discussion Papers 2022-12, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    13. Cornaglia, Francesca & Drouvelis, Michalis & Masella, Paolo, 2019. "Competition and the role of group identity," Journal of Economic Behavior & Organization, Elsevier, vol. 162(C), pages 136-145.
    14. Gantner, Anita & Horn, Kristian & Kerschbamer, Rudolf, 2016. "Fair and efficient division through unanimity bargaining when claims are subjective," Journal of Economic Psychology, Elsevier, vol. 57(C), pages 56-73.
    15. Buser, Thomas & Ranehill, Eva & van Veldhuizen, Roel, 2021. "Gender differences in willingness to compete: The role of public observability," Journal of Economic Psychology, Elsevier, vol. 83(C).
    16. Ro’i Zultan & Eldar Dadon, 2023. "Missing the forest for the trees: when monitoring quantitative measures distorts task prioritization," Working Papers 2319, Ben-Gurion University of the Negev, Department of Economics.
    17. Casari, Marco & Tavoni, Alessandro, 2024. "Climate clubs in the laboratory," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 110(C).
    18. Martin G. Kocher & Fangfang Tan & Jing Yu, 2018. "Providing Global Public Goods: Electoral Delegation And Cooperation," Economic Inquiry, Western Economic Association International, vol. 56(1), pages 381-397, January.
    19. Kessel, Dany & Mollerstrom, Johanna & van Veldhuizen, Roel, 2021. "Can simple advice eliminate the gender gap in willingness to compete?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 138, pages 1-1.
    20. Moradi, Homayoon, 2018. "Selfless ignorance: Too good to be true," Discussion Papers, Research Unit: Market Behavior SP II 2018-208, WZB Berlin Social Science Center.
    21. Fischbacher, Urs & Kübler, Dorothea & Stüber, Robert, 2022. "Betting on diversity: Occupational segregation and gender stereotypes," Discussion Papers, Research Unit: Market Behavior SP II 2022-207, WZB Berlin Social Science Center.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:dpr:wpaper:1233r. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/isosujp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.