IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2412.18337.html
   My bibliography  Save this paper

The Value of AI-Generated Metadata for UGC Platforms: Evidence from a Large-scale Field Experiment

Author

Listed:
  • Xinyi Zhang
  • Chenshuo Sun
  • Renyu Zhang
  • Khim-Yong Goh

Abstract

AI-generated content (AIGC), such as advertisement copy, product descriptions, and social media posts, is becoming ubiquitous in business practices. However, the value of AI-generated metadata, such as titles, remains unclear on user-generated content (UGC) platforms. To address this gap, we conducted a large-scale field experiment on a leading short-video platform in Asia to provide about 1 million users access to AI-generated titles for their uploaded videos. Our findings show that the provision of AI-generated titles significantly boosted content consumption, increasing valid watches by 1.6% and watch duration by 0.9%. When producers adopted these titles, these increases jumped to 7.1% and 4.1%, respectively. This viewership-boost effect was largely attributed to the use of this generative AI (GAI) tool increasing the likelihood of videos having a title by 41.4%. The effect was more pronounced for groups more affected by metadata sparsity. Mechanism analysis revealed that AI-generated metadata improved user-video matching accuracy in the platform's recommender system. Interestingly, for a video for which the producer would have posted a title anyway, adopting the AI-generated title decreased its viewership on average, implying that AI-generated titles may be of lower quality than human-generated ones. However, when producers chose to co-create with GAI and significantly revised the AI-generated titles, the videos outperformed their counterparts with either fully AI-generated or human-generated titles, showcasing the benefits of human-AI co-creation. This study highlights the value of AI-generated metadata and human-AI metadata co-creation in enhancing user-content matching and content consumption for UGC platforms.

Suggested Citation

  • Xinyi Zhang & Chenshuo Sun & Renyu Zhang & Khim-Yong Goh, 2024. "The Value of AI-Generated Metadata for UGC Platforms: Evidence from a Large-scale Field Experiment," Papers 2412.18337, arXiv.org.
    • Handle: RePEc:arx:papers:2412.18337
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2412.18337
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chiara Longoni & Andrea Bonezzi & Carey K Morewedge, 2019. "Resistance to Medical Artificial Intelligence," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 46(4), pages 629-650.
    2. Maytal Saar-Tsechansky & Prem Melville & Foster Provost, 2009. "Active Feature-Value Acquisition," Management Science, INFORMS, vol. 55(4), pages 664-684, April.
    3. Ni Huang & Probal Mojumder & Tianshu Sun & Jinchi Lv & Joseph M. Golden, 2021. "Not Registered? Please Sign Up First: A Randomized Field Experiment on the Ex Ante Registration Request," Information Systems Research, INFORMS, vol. 32(3), pages 914-931, September.
    4. Ruomeng Cui & Jun Li & Dennis J. Zhang, 2020. "Reducing Discrimination with Reviews in the Sharing Economy: Evidence from Field Experiments on Airbnb," Management Science, INFORMS, vol. 66(3), pages 1071-1094, March.
    5. Jiankun Sun & Dennis J. Zhang & Haoyuan Hu & Jan A. Van Mieghem, 2022. "Predicting Human Discretion to Adjust Algorithmic Prescription: A Large-Scale Field Experiment in Warehouse Operations," Management Science, INFORMS, vol. 68(2), pages 846-865, February.
    6. Zikun Ye & Dennis J. Zhang & Heng Zhang & Renyu Zhang & Xin Chen & Zhiwei Xu, 2023. "Cold Start to Improve Market Thickness on Online Advertising Platforms: Data-Driven Algorithms and Field Experiments," Management Science, INFORMS, vol. 69(7), pages 3838-3860, July.
    7. Saravanan Kesavan & Tarun Kushwaha, 2020. "Field Experiment on the Profit Implications of Merchants’ Discretionary Power to Override Data-Driven Decision-Making Tools," Management Science, INFORMS, vol. 66(11), pages 5182-5190, November.
    8. Martin Reisenbichler & Thomas Reutterer & David A. Schweidel & Daniel Dan, 2022. "Frontiers: Supporting Content Marketing with Natural Language Generation," Marketing Science, INFORMS, vol. 41(3), pages 441-452, May.
    9. Mikhail Lysyakov & Siva Viswanathan, 2023. "Threatened by AI: Analyzing Users’ Responses to the Introduction of AI in a Crowd-Sourcing Platform," Information Systems Research, INFORMS, vol. 34(3), pages 1191-1210, September.
    10. Tianshu Sun & Lanfei Shi & Siva Viswanathan & Elena Zheleva, 2019. "Motivating Effective Mobile App Adoptions: Evidence from a Large-Scale Randomized Field Experiment," Information Systems Research, INFORMS, vol. 30(2), pages 523-539, June.
    11. Xingyue (Luna) Zhang & James A. Dearden & Yuliang Yao, 2022. "Let them stay or let them go? Online retailer pricing strategy for managing stockouts," Production and Operations Management, Production and Operations Management Society, vol. 31(11), pages 4173-4190, November.
    12. Zhen Fang & Ming Fan & Apurva Jain, 2023. "Content proliferation and narrowcasting in the age of streaming media," Production and Operations Management, Production and Operations Management Society, vol. 32(10), pages 3295-3310, October.
    13. Dandan Qiao & Shun-Yang Lee & Andrew B. Whinston & Qiang Wei, 2020. "Financial Incentives Dampen Altruism in Online Prosocial Contributions: A Study of Online Reviews," Information Systems Research, INFORMS, vol. 31(4), pages 1361-1375, December.
    14. Sriram Narayanan & Sridhar Balasubramanian & Jayashankar M. Swaminathan, 2009. "A Matter of Balance: Specialization, Task Variety, and Individual Learning in a Software Maintenance Environment," Management Science, INFORMS, vol. 55(11), pages 1861-1876, November.
    15. Ni Huang & Gordon Burtch & Bin Gu & Yili Hong & Chen Liang & Kanliang Wang & Dongpu Fu & Bo Yang, 2019. "Motivating User-Generated Content with Performance Feedback: Evidence from Randomized Field Experiments," Management Science, INFORMS, vol. 65(1), pages 327-345, January.
    16. Cole, Rebel A. & Sokolyk, Tatyana, 2018. "Debt financing, survival, and growth of start-up firms," Journal of Corporate Finance, Elsevier, vol. 50(C), pages 609-625.
    17. Scott C. Ellis & Shashank Rao & Dheeraj Raju & Thomas J. Goldsby, 2018. "RFID Tag Performance: Linking the Laboratory to the Field through Unsupervised Learning," Production and Operations Management, Production and Operations Management Society, vol. 27(10), pages 1834-1848, October.
    18. Gordon Burtch & Qinglai He & Yili Hong & Dokyun Lee, 2022. "How Do Peer Awards Motivate Creative Content? Experimental Evidence from Reddit," Management Science, INFORMS, vol. 68(5), pages 3488-3506, May.
    19. Jiaxu Peng & Jungpil Hahn & Ke-Wei Huang, 2023. "Handling Missing Values in Information Systems Research: A Review of Methods and Assumptions," Information Systems Research, INFORMS, vol. 34(1), pages 5-26, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Siliang Tong & Nan Jia & Xueming Luo & Zheng Fang, 2021. "The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance," Strategic Management Journal, Wiley Blackwell, vol. 42(9), pages 1600-1631, September.
    2. Brüns, Jasper David & Meißner, Martin, 2024. "Do you create your content yourself? Using generative artificial intelligence for social media content creation diminishes perceived brand authenticity," Journal of Retailing and Consumer Services, Elsevier, vol. 79(C).
    3. Sulin Ba & Shu He & Shun‐Yang Lee, 2022. "Mobile App Adoption and Its Differential Impact on Consumer Shopping Behavior," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 764-780, February.
    4. Yipu Deng & Jinyang Zheng & Warut Khern-am-nuai & Karthik Kannan, 2022. "More than the Quantity: The Value of Editorial Reviews for a User-Generated Content Platform," Management Science, INFORMS, vol. 68(9), pages 6865-6888, September.
    5. Gloria Urrea & Eunae Yoo, 2023. "The role of volunteer experience on performance on online volunteering platforms," Production and Operations Management, Production and Operations Management Society, vol. 32(2), pages 416-433, February.
    6. Yingda Lu & Youwei Wang & Yuxin Chen & Yun Xiong, 2023. "The role of data‐based intelligence and experience on time efficiency of taxi drivers: An empirical investigation using large‐scale sensor data," Production and Operations Management, Production and Operations Management Society, vol. 32(11), pages 3665-3682, November.
    7. Angelopoulos, Spyros & Bendoly, Elliot & Fransoo, Jan C. & Hoberg, Kai & Ou, Carol & Tenhiälä, Antti, 2023. "Digital transformation in operations management: Fundamental change through agency reversal," Other publications TiSEM 373742f5-0b87-4276-9ed6-8, Tilburg University, School of Economics and Management.
    8. Maria De‐Arteaga & Stefan Feuerriegel & Maytal Saar‐Tsechansky, 2022. "Algorithmic fairness in business analytics: Directions for research and practice," Production and Operations Management, Production and Operations Management Society, vol. 31(10), pages 3749-3770, October.
    9. Hermann, Erik & Puntoni, Stefano, 2024. "Artificial intelligence and consumer behavior: From predictive to generative AI," Journal of Business Research, Elsevier, vol. 180(C).
    10. Chen, Changdong, 2024. "How consumers respond to service failures caused by algorithmic mistakes: The role of algorithmic interpretability," Journal of Business Research, Elsevier, vol. 176(C).
    11. Tinglong Dai & Sridhar Tayur, 2022. "Designing AI‐augmented healthcare delivery systems for physician buy‐in and patient acceptance," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4443-4451, December.
    12. Leah Warfield Smith & Randall Lee Rose & Alex R. Zablah & Heath McCullough & Mohammad “Mike” Saljoughian, 2023. "Examining post-purchase consumer responses to product automation," Journal of the Academy of Marketing Science, Springer, vol. 51(3), pages 530-550, May.
    13. Jan Ole Krugmann & Jochen Hartmann, 2024. "Sentiment Analysis in the Age of Generative AI," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 11(1), pages 1-19, December.
    14. Nik Hadiyan Binti Nik Azman, 2023. "Game analysis between startup and banks," GATR Journals gjbssr631, Global Academy of Training and Research (GATR) Enterprise.
    15. S. Alex Yang & Angela Huyue Zhang, 2024. "Generative AI and Copyright: A Dynamic Perspective," Papers 2402.17801, arXiv.org.
    16. Diwas Singh KC & Bradley R. Staats, 2012. "Accumulating a Portfolio of Experience: The Effect of Focal and Related Experience on Surgeon Performance," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 618-633, October.
    17. Zhepeng Li & Xiao Fang & Xue Bai & Olivia R. Liu Sheng, 2017. "Utility-Based Link Recommendation for Online Social Networks," Management Science, INFORMS, vol. 63(6), pages 1938-1952, June.
    18. Cloos, Janis & Mohr, Svenja, 2022. "Acceptance of data sharing in smartphone apps from key industries of the digital transformation: A representative population survey for Germany," Technological Forecasting and Social Change, Elsevier, vol. 176(C).
    19. Chamaret, Cécile & Steyer, Véronique & Mayer, Julie C., 2020. "“Hands off my meter!” when municipalities resist smart meters: Linking arguments and degrees of resistance," Energy Policy, Elsevier, vol. 144(C).
    20. Pengkun Wu & Eric W. T. Ngai & Yuanyuan Wu, 2023. "Impact of praise cashback strategy: Implications for consumers and e‐businesses," Production and Operations Management, Production and Operations Management Society, vol. 32(9), pages 2825-2845, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2412.18337. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.