IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v205y2024ics0040162524002841.html
   My bibliography  Save this article

AI-empowered scale development: Testing the potential of ChatGPT

Author

Listed:
  • Hoffmann, Stefan
  • Lasarov, Wassili
  • Dwivedi, Yogesh K.

Abstract

AI-tools such as ChatGPT can assist researchers to improve the performance of the research process. This paper examines whether researchers could apply ChatGPT to develop and empirically validate new research scales. The study describes a process how to prompt ChatGPT to assist the scale development of a new construct, using the example of the construct of perceived value of ChatGPT-supported consumer behavior. The paper reports four main empirical studies (US: N = 148; Australia: N = 317; UK: N = 108; Germany: N = 51) that have been employed to validate the newly developed scale. The first study purifies the scale. The following studies confirm the adjusted factorial validity of the reduced scale. Although the empirical data imply a simplification of the initial multi-dimensional scale, the final three-dimensional operationalization is highly reliable and valid. The paper outlines the shortcomings and several critical notes to stimulate more research and discussion in this area.

Suggested Citation

  • Hoffmann, Stefan & Lasarov, Wassili & Dwivedi, Yogesh K., 2024. "AI-empowered scale development: Testing the potential of ChatGPT," Technological Forecasting and Social Change, Elsevier, vol. 205(C).
    • Handle: RePEc:eee:tefoso:v:205:y:2024:i:c:s0040162524002841
    DOI: 10.1016/j.techfore.2024.123488
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0040162524002841
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2024.123488?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dalvi-Esfahani, Mohammad & Mosharaf-Dehkordi, Mehdi & Leong, Lam Wai & Ramayah, T. & Jamal Kanaan-Jebna, Abdulkarim M., 2023. "Exploring the drivers of XAI-enhanced clinical decision support systems adoption: Insights from a stimulus-organism-response perspective," Technological Forecasting and Social Change, Elsevier, vol. 195(C).
    2. Loureiro, Sandra Maria Correia & Guerreiro, João & Tussyadiah, Iis, 2021. "Artificial intelligence in business: State of the art and future research agenda," Journal of Business Research, Elsevier, vol. 129(C), pages 911-926.
    3. Dwivedi, Yogesh K. & Hughes, Laurie & Ismagilova, Elvira & Aarts, Gert & Coombs, Crispin & Crick, Tom & Duan, Yanqing & Dwivedi, Rohita & Edwards, John & Eirug, Aled & Galanos, Vassilis & Ilavarasan, , 2021. "Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy," International Journal of Information Management, Elsevier, vol. 57(C).
    4. Chris Stokel-Walker, 2023. "ChatGPT listed as author on research papers: many scientists disapprove," Nature, Nature, vol. 613(7945), pages 620-621, January.
    5. Shi, Baisheng & Wang, Hao, 2023. "An AI-enabled approach for improving advertising identification and promotion in social networks," Technological Forecasting and Social Change, Elsevier, vol. 188(C).
    6. Vlačić, Božidar & Corbo, Leonardo & Costa e Silva, Susana & Dabić, Marina, 2021. "The evolving role of artificial intelligence in marketing: A review and research agenda," Journal of Business Research, Elsevier, vol. 128(C), pages 187-203.
    7. Baabdullah, Abdullah M. & Alalwan, Ali Abdallah & Algharabat, Raed S. & Metri, Bhimaraya & Rana, Nripendra P., 2022. "Virtual agents and flow experience: An empirical examination of AI-powered chatbots," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    8. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    9. Nisreen Ameen & Ali Tarhini & Alexander Reppel & Amitabh Anand, 2021. "Customer experiences in the age of artificial intelligence," Post-Print halshs-03045430, HAL.
    10. Yogesh K. Dwivedi & A. Sharma & Nripendra P. Rana & M. Giannakis & P. Goel & Vincent Dutot, 2023. "Evolution of Artificial Intelligence Research in Technological Forecasting and Social Change: Research Topics, Trends, and Future Directions," Post-Print hal-04292607, HAL.
    11. Mekhail Mustak & Joni Salminen & Loïc Plé & Jochen Wirtz, 2021. "Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda," Post-Print hal-03269994, HAL.
    12. Matthew Hutson, 2022. "Could AI help you to write your next paper?," Nature, Nature, vol. 611(7934), pages 192-193, November.
    13. Chaturvedi, Rijul & Verma, Sanjeev & Das, Ronnie & Dwivedi, Yogesh K., 2023. "Social companionship with artificial intelligence: Recent trends and future avenues," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    14. Fernandes, Teresa & Oliveira, Elisabete, 2021. "Understanding consumers’ acceptance of automated technologies in service encounters: Drivers of digital voice assistants adoption," Journal of Business Research, Elsevier, vol. 122(C), pages 180-191.
    15. Makarius, Erin E. & Mukherjee, Debmalya & Fox, Joseph D. & Fox, Alexa K., 2020. "Rising with the machines: A sociotechnical framework for bringing artificial intelligence into the organization," Journal of Business Research, Elsevier, vol. 120(C), pages 262-273.
    16. Mustak, Mekhail & Salminen, Joni & Plé, Loïc & Wirtz, Jochen, 2021. "Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda," Journal of Business Research, Elsevier, vol. 124(C), pages 389-404.
    17. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    18. William Black & Barry J. Babin, 2019. "Multivariate Data Analysis: Its Approach, Evolution, and Impact," Springer Books, in: Barry J. Babin & Marko Sarstedt (ed.), The Great Facilitator, pages 121-130, Springer.
    19. Shen, Guorong, 2022. "AI-enabled talent training for the cross-cultural news communication talent," Technological Forecasting and Social Change, Elsevier, vol. 185(C).
    20. Füller, Johann & Hutter, Katja & Wahl, Julian & Bilgram, Volker & Tekic, Zeljko, 2022. "How AI revolutionizes innovation management – Perceptions and implementation preferences of AI-based innovators," Technological Forecasting and Social Change, Elsevier, vol. 178(C).
    21. Fuller, Christie M. & Simmering, Marcia J. & Atinc, Guclu & Atinc, Yasemin & Babin, Barry J., 2016. "Common methods variance detection in business research," Journal of Business Research, Elsevier, vol. 69(8), pages 3192-3198.
    22. Trocin, Cristina & Hovland, Ingrid Våge & Mikalef, Patrick & Dremel, Christian, 2021. "How Artificial Intelligence affords digital innovation: A cross-case analysis of Scandinavian companies," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    23. Wang, Yichuan & Kung, LeeAnn & Byrd, Terry Anthony, 2018. "Big data analytics: Understanding its capabilities and potential benefits for healthcare organizations," Technological Forecasting and Social Change, Elsevier, vol. 126(C), pages 3-13.
    24. Peres, Renana & Schreier, Martin & Schweidel, David & Sorescu, Alina, 2023. "On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice," International Journal of Research in Marketing, Elsevier, vol. 40(2), pages 269-275.
    25. Dwivedi, Yogesh K. & Sharma, Anuj & Rana, Nripendra P. & Giannakis, Mihalis & Goel, Pooja & Dutot, Vincent, 2023. "Evolution of artificial intelligence research in Technological Forecasting and Social Change: Research topics, trends, and future directions," Technological Forecasting and Social Change, Elsevier, vol. 192(C).
    26. repec:ucp:bkecon:9780226316529 is not listed on IDEAS
    27. Short, Cole E. & Short, Jeremy C., 2023. "The artificially intelligent entrepreneur: ChatGPT, prompt engineering, and entrepreneurial rhetoric creation," Journal of Business Venturing Insights, Elsevier, vol. 19(C).
    28. Li, Jian & Huang, Jin-Song, 2020. "Dimensions of artificial intelligence anxiety based on the integrated fear acquisition theory," Technology in Society, Elsevier, vol. 63(C).
    29. ., 2023. "Harnessing diversity in teams," Chapters, in: The Global Challenge, chapter 9, pages 278-313, Edward Elgar Publishing.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hermann, Erik & Puntoni, Stefano, 2024. "Artificial intelligence and consumer behavior: From predictive to generative AI," Journal of Business Research, Elsevier, vol. 180(C).
    2. wael AL-khatib, Ayman, 2023. "Drivers of generative artificial intelligence to fostering exploitative and exploratory innovation: A TOE framework," Technology in Society, Elsevier, vol. 75(C).
    3. Wenkai Zhou & Chi Zhang & Linwan Wu & Meghana Shashidhar, 2023. "ChatGPT and marketing: Analyzing public discourse in early Twitter posts," Journal of Marketing Analytics, Palgrave Macmillan, vol. 11(4), pages 693-706, December.
    4. Yogesh K. Dwivedi & A. Sharma & Nripendra P. Rana & M. Giannakis & P. Goel & Vincent Dutot, 2023. "Evolution of Artificial Intelligence Research in Technological Forecasting and Social Change: Research Topics, Trends, and Future Directions," Post-Print hal-04292607, HAL.
    5. Shahzad, Fakhar & Zaied, Younes Ben & Shahzad, Muhammad Asim & Mahmood, Faisal, 2024. "Insights into the performance of green supply chain in the Chinese semiconductor industry," International Journal of Production Economics, Elsevier, vol. 273(C).
    6. Fosso Wamba, Samuel & Queiroz, Maciel M. & Chiappetta Jabbour, Charbel Jose & Shi, Chunming (Victor), 2023. "Are both generative AI and ChatGPT game changers for 21st-Century operations and supply chain excellence?," International Journal of Production Economics, Elsevier, vol. 265(C).
    7. O. C. Ferrell & Dana E. Harrison & Linda K. Ferrell & Haya Ajjan & Bryan W. Hochstein, 2024. "A theoretical framework to guide AI ethical decision making," AMS Review, Springer;Academy of Marketing Science, vol. 14(1), pages 53-67, June.
    8. Broekhuizen, Thijs & Dekker, Henri & de Faria, Pedro & Firk, Sebastian & Nguyen, Dinh Khoi & Sofka, Wolfgang, 2023. "AI for managing open innovation: Opportunities, challenges, and a research agenda," Journal of Business Research, Elsevier, vol. 167(C).
    9. Erik Hermann, 2022. "Leveraging Artificial Intelligence in Marketing for Social Good—An Ethical Perspective," Journal of Business Ethics, Springer, vol. 179(1), pages 43-61, August.
    10. Chakraborty, Debarun & Polisetty, Aruna & G, Sowmya & Rana, Nripendra P. & Khorana, Sangeeta, 2024. "Unlocking the potential of AI: Enhancing consumer engagement in the beauty and cosmetic product purchases," Journal of Retailing and Consumer Services, Elsevier, vol. 79(C).
    11. Mariani, Marcello M. & Hashemi, Novin & Wirtz, Jochen, 2023. "Artificial intelligence empowered conversational agents: A systematic literature review and research agenda," Journal of Business Research, Elsevier, vol. 161(C).
    12. Volkmar, Gioia & Fischer, Peter M. & Reinecke, Sven, 2022. "Artificial Intelligence and Machine Learning: Exploring drivers, barriers, and future developments in marketing management," Journal of Business Research, Elsevier, vol. 149(C), pages 599-614.
    13. Goodell, John W. & Kumar, Satish & Lim, Weng Marc & Pattnaik, Debidutta, 2021. "Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    14. Ali, Omar & Murray, Peter A. & Momin, Mujtaba & Dwivedi, Yogesh K. & Malik, Tegwen, 2024. "The effects of artificial intelligence applications in educational settings: Challenges and strategies," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    15. Chatterjee, Sheshadri & Chaudhuri, Ranjan & Vrontis, Demetris & Jabeen, Fauzia, 2022. "Digital transformation of organization using AI-CRM: From microfoundational perspective with leadership support," Journal of Business Research, Elsevier, vol. 153(C), pages 46-58.
    16. Blasco-Arcas, Lorena & Lee, Hsin-Hsuan Meg & Kastanakis, Minas N. & Alcañiz, Mariano & Reyes-Menendez, Ana, 2022. "The role of consumer data in marketing: A research agenda," Journal of Business Research, Elsevier, vol. 146(C), pages 436-452.
    17. Battisti, Sandro & Agarwal, Nivedita & Brem, Alexander, 2022. "Creating new tech entrepreneurs with digital platforms: Meta-organizations for shared value in data-driven retail ecosystems," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    18. Donthu, Naveen & Kumar, Satish & Mukherjee, Debmalya & Pandey, Nitesh & Lim, Weng Marc, 2021. "How to conduct a bibliometric analysis: An overview and guidelines," Journal of Business Research, Elsevier, vol. 133(C), pages 285-296.
    19. Mariani, Marcello M. & Machado, Isa & Nambisan, Satish, 2023. "Types of innovation and artificial intelligence: A systematic quantitative literature review and research agenda," Journal of Business Research, Elsevier, vol. 155(PB).
    20. Mariani, Marcello M. & Machado, Isa & Magrelli, Vittoria & Dwivedi, Yogesh K., 2023. "Artificial intelligence in innovation research: A systematic review, conceptual framework, and future research directions," Technovation, Elsevier, vol. 122(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:205:y:2024:i:c:s0040162524002841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.