IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v65y2019i1p327-345.html
   My bibliography  Save this article

Motivating User-Generated Content with Performance Feedback: Evidence from Randomized Field Experiments

Author

Listed:
  • Ni Huang

    (W. P. Carey School of Business, Arizona State University, Tempe, Arizona 85287)

  • Gordon Burtch

    (Carlson School of Management, University of Minnesota, Minneapolis, Minnesota 55455)

  • Bin Gu

    (W. P. Carey School of Business, Arizona State University, Tempe, Arizona 85287)

  • Yili Hong

    (W. P. Carey School of Business, Arizona State University, Tempe, Arizona 85287)

  • Chen Liang

    (W. P. Carey School of Business, Arizona State University, Tempe, Arizona 85287)

  • Kanliang Wang

    (School of Business, Renmin University of China, Beijing 100872, China)

  • Dongpu Fu

    (School of Information, Capital University of Economics and Business, Beijing 100070, China)

  • Bo Yang

    (School of Information, Renmin University of China, Beijing 100872, China)

Abstract

We design a series of online performance feedback interventions that aim to motivate the production of user-generated content (UGC). Drawing on social value orientation (SVO) theory, we develop a novel set of alternative feedback message framings, aligned with cooperation (e.g., your content benefited others), individualism (e.g., your content was of high quality), and competition (e.g., your content was better than others). We hypothesize how gender (a proxy for SVO) moderates response to each framing, and we report on two randomized experiments, one in partnership with a mobile-app–based recipe crowdsourcing platform, and a follow-up experiment on Amazon Mechanical Turk involving an ideation task. We find evidence that cooperatively framed feedback is most effective for motivating female subjects, whereas competitively framed feedback is most effective at motivating male subjects. Our work contributes to the literatures on performance feedback and UGC production by introducing cooperative performance feedback as a theoretically motivated, novel intervention that speaks directly to users’ altruistic intent in a variety of task settings. Our work also contributes to the message-framing literature in considering competition as a novel addition to the altruism–egoism dichotomy oft explored in public good settings.

Suggested Citation

  • Ni Huang & Gordon Burtch & Bin Gu & Yili Hong & Chen Liang & Kanliang Wang & Dongpu Fu & Bo Yang, 2019. "Motivating User-Generated Content with Performance Feedback: Evidence from Randomized Field Experiments," Management Science, INFORMS, vol. 65(1), pages 327-345, January.
  • Handle: RePEc:inm:ormnsc:v:65:y:2019:i:1:p:327-345
    DOI: 10.1287/mnsc.2017.2944
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/mnsc.2017.2944
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mnsc.2017.2944?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Young Kwark & Jianqing Chen & Srinivasan Raghunathan, 2014. "Online Product Reviews: Implications for Retailers and Competing Manufacturers," Information Systems Research, INFORMS, vol. 25(1), pages 93-110, March.
    2. Tanjim Hossain & John A. List, 2012. "The Behavioralist Visits the Factory: Increasing Productivity Using Simple Framing Manipulations," Management Science, INFORMS, vol. 58(12), pages 2151-2167, December.
    3. Olga Shurchkov, 2012. "Under Pressure: Gender Differences In Output Quality And Quantity Under Competition And Time Constraints," Journal of the European Economic Association, European Economic Association, vol. 10(5), pages 1189-1213, October.
    4. James Andreoni & Lise Vesterlund, 2001. "Which is the Fair Sex? Gender Differences in Altruism," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(1), pages 293-312.
    5. Luís Cabral & Lingfang (Ivy) Li, 2015. "A Dollar for Your Thoughts: Feedback-Conditional Rebates on eBay," Management Science, INFORMS, vol. 61(9), pages 2052-2063, September.
    6. Jae Yun Moon & Lee S. Sproull, 2008. "The Role of Feedback in Managing the Internet-Based Volunteer Work Force," Information Systems Research, INFORMS, vol. 19(4), pages 494-515, December.
    7. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    8. Louis-Philippe Morin, 2015. "Do Men and Women Respond Differently to Competition? Evidence from a Major Education Reform," Journal of Labor Economics, University of Chicago Press, vol. 33(2), pages 443-491.
    9. David L. Dickinson & Jill Tiefenthaler, 2002. "What Is Fair? Experimental Evidence," Southern Economic Journal, John Wiley & Sons, vol. 69(2), pages 414-428, October.
    10. Nabanita Datta Gupta & Anders Poulsen & Marie Claire Villeval, 2013. "Gender Matching And Competitiveness: Experimental Evidence," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 816-835, January.
    11. Gordon Burtch & Yili Hong & Ravi Bapna & Vladas Griskevicius, 2018. "Stimulating Online Reviews by Combining Financial Incentives and Social Norms," Management Science, INFORMS, vol. 64(5), pages 2065-2082, May.
    12. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    13. Jeffrey A. Roberts & Il-Horn Hann & Sandra A. Slaughter, 2006. "Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects," Management Science, INFORMS, vol. 52(7), pages 984-999, July.
    14. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    15. Dufwenberg, Martin & Muren, Astri, 2006. "Gender composition in teams," Journal of Economic Behavior & Organization, Elsevier, vol. 61(1), pages 50-54, September.
    16. Chee Wei Phang & Atreyi Kankanhalli & Bernard C. Y. Tan, 2015. "What Motivates Contributors vs. Lurkers? An Investigation of Online Feedback Forums," Information Systems Research, INFORMS, vol. 26(4), pages 773-792, December.
    17. Yan Chen & F. Maxwell Harper & Joseph Konstan & Sherry Xin Li, 2010. "Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens," American Economic Review, American Economic Association, vol. 100(4), pages 1358-1398, September.
    18. Patrick J. Bateman & Peter H. Gray & Brian S. Butler, 2011. "Research Note ---The Impact of Community Commitment on Participation in Online Communities," Information Systems Research, INFORMS, vol. 22(4), pages 841-854, December.
    19. Chen Zhang & Jungpil Hahn & Prabuddha De, 2013. "Research Note ---Continued Participation in Online Innovation Communities: Does Community Response Matter Equally for Everyone?," Information Systems Research, INFORMS, vol. 24(4), pages 1112-1130, December.
    20. Daniel Jones & Sera Linardi, 2014. "Wallflowers: Experimental Evidence of an Aversion to Standing Out," Management Science, INFORMS, vol. 60(7), pages 1757-1771, July.
    21. Uri Gneezy & Muriel Niederle & Aldo Rustichini, 2003. "Performance in Competitive Environments: Gender Differences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 1049-1074.
    22. Olivier Toubia & Andrew T. Stephen, 2013. "Intrinsic vs. Image-Related Utility in Social Media: Why Do People Contribute Content to Twitter?," Marketing Science, INFORMS, vol. 32(3), pages 368-392, May.
    23. Xiaohua Zeng & Liyuan Wei, 2013. "Social Ties and User Content Generation: Evidence from Flickr," Information Systems Research, INFORMS, vol. 24(1), pages 71-87, March.
    24. Paulo B. Goes & Chenhui Guo & Mingfeng Lin, 2016. "Do Incentive Hierarchies Induce User Effort? Evidence from an Online Knowledge Exchange," Information Systems Research, INFORMS, vol. 27(3), pages 497-516, September.
    25. Robert J. Fisher & Mark Vandenbosch & Kersi D. Antia, 2008. "An Empathy-Helping Perspective on Consumers' Responses to Fund-Raising Appeals," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 35(3), pages 519-531, February.
    26. Wozniak, David, 2012. "Gender differences in a market with relative performance feedback: Professional tennis players," Journal of Economic Behavior & Organization, Elsevier, vol. 83(1), pages 158-171.
    27. De Paola, Maria & Gioia, Francesca & Scoppa, Vincenzo, 2014. "Overconfidence, omens and gender heterogeneity: Results from a field experiment," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 237-252.
    28. Xiaoquan (Michael) Zhang & Feng Zhu, 2011. "Group Size and Incentives to Contribute: A Natural Experiment at Chinese Wikipedia," American Economic Review, American Economic Association, vol. 101(4), pages 1601-1615, June.
    29. David Godes & José C. Silva, 2012. "Sequential and Temporal Dynamics of Online Opinion," Marketing Science, INFORMS, vol. 31(3), pages 448-473, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Feng Zhu & Xinxin Li & Ehsan Valavi & Marco Iansiti, 2021. "Network Interconnectivity and Entry into Platform Markets," Information Systems Research, INFORMS, vol. 32(3), pages 1009-1024, September.
    2. Yipu Deng & Jinyang Zheng & Warut Khern-am-nuai & Karthik Kannan, 2022. "More than the Quantity: The Value of Editorial Reviews for a User-Generated Content Platform," Management Science, INFORMS, vol. 68(9), pages 6865-6888, September.
    3. Carlos Fernández-Loría & Maxime C. Cohen & Anindya Ghose, 2023. "Evolution of Referrals over Customers’ Life Cycle: Evidence from a Ride-Sharing Platform," Information Systems Research, INFORMS, vol. 34(2), pages 698-720, June.
    4. Hernandez-Arenaz, Iñigo, 2020. "Stereotypes and tournament self-selection: A theoretical and experimental approach," European Economic Review, Elsevier, vol. 126(C).
    5. Zaiyan Wei & Mo Xiao & Rong Rong, 2021. "Network Size and Content Generation on Social Media Platforms," Production and Operations Management, Production and Operations Management Society, vol. 30(5), pages 1406-1426, May.
    6. Yun Young Hur & Fujie Jin & Xitong Li & Yuan Cheng & Yu Jeffrey Hu, 2023. "Does Social Influence Change with Other Information Sources? A Large-Scale Randomized Experiment in Medical Crowdfunding," Information Systems Research, INFORMS, vol. 34(4), pages 1476-1492, December.
    7. Chenhui (Julian) Guo & Tae Hun Kim & Anjana Susarla & Vallabh Sambamurthy, 2020. "Understanding Content Contribution Behavior in a Geosegmented Mobile Virtual Community: The Context of Waze," Information Systems Research, INFORMS, vol. 31(4), pages 1398-1420, December.
    8. Kimmy Wa Chan & Stella Yiyan Li & Jian Ni & John JianJun Zhu, 2021. "What Feedback Matters? The Role of Experience in Motivating Crowdsourcing Innovation," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 103-126, January.
    9. Maria Marchenko & Hendrik Sonnabend, 2022. "The Never Ending Book: The role of external stimuli and peer feedback in user-generated content production," Department of Economics Working Papers wuwp320, Vienna University of Economics and Business, Department of Economics.
    10. Lucian Visinescu & Nicholas Evangelopoulos, 2024. "Designing adaptive feedback mechanisms with text mining capabilities: An illustration on eBay," Electronic Markets, Springer;IIM University of St. Gallen, vol. 34(1), pages 1-21, December.
    11. Dena Yadin & Inbal Yahav & Lior Zalmanson & Nira Munichor, 2024. "Resolving the Ethical Tension Between Creating a Civil Environment and Facilitating Free Expression Online: Comment Reordering as an Alternative to Comment Moderation," Journal of Business Ethics, Springer, vol. 192(2), pages 261-283, June.
    12. Wei, Zihan & Zhang, Mingli & Qiao, Tong, 2022. "Effect of personal branding stereotypes on user engagement on short-video platforms," Journal of Retailing and Consumer Services, Elsevier, vol. 69(C).
    13. Ni Huang & Gordon Burtch & Yumei He & Yili Hong, 2022. "Managing Congestion in a Matching Market via Demand Information Disclosure," Information Systems Research, INFORMS, vol. 33(4), pages 1196-1220, December.
    14. Ni Huang & Jiayin Zhang & Gordon Burtch & Xitong Li & Peiyu Chen, 2021. "Combating Procrastination on Massive Online Open Courses via Optimal Calls to Action," Information Systems Research, INFORMS, vol. 32(2), pages 301-317, June.
    15. Tianshu Sun & Siva Viswanathan & Elena Zheleva, 2021. "Creating Social Contagion Through Firm-Mediated Message Design: Evidence from a Randomized Field Experiment," Management Science, INFORMS, vol. 67(2), pages 808-827, February.
    16. Heeseung Andrew Lee & Angela Aerry Choi & Tianshu Sun & Wonseok Oh, 2021. "Reviewing Before Reading? An Empirical Investigation of Book-Consumption Patterns and Their Effects on Reviews and Sales," Information Systems Research, INFORMS, vol. 32(4), pages 1368-1389, December.
    17. Ji Wu & Zhiqiang (Eric) Zheng & J. Leon Zhao, 2021. "FairPlay: Detecting and Deterring Online Customer Misbehavior," Information Systems Research, INFORMS, vol. 32(4), pages 1323-1346, December.
    18. Michael Rivera & Liangfei Qiu & Subodha Kumar & Tony Petrucci, 2021. "Are Traditional Performance Reviews Outdated? An Empirical Analysis on Continuous, Real-Time Feedback in the Workplace," Information Systems Research, INFORMS, vol. 32(2), pages 517-540, June.
    19. Gordon Burtch & Qinglai He & Yili Hong & Dokyun Lee, 2022. "How Do Peer Awards Motivate Creative Content? Experimental Evidence from Reddit," Management Science, INFORMS, vol. 68(5), pages 3488-3506, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hinnosaar, Marit, 2019. "Gender inequality in new media: Evidence from Wikipedia," Journal of Economic Behavior & Organization, Elsevier, vol. 163(C), pages 262-276.
    2. Becchetti, Leonardo & Degli Antoni, Giacomo & Ottone, Stefania & Solferino, Nazaria, 2013. "Allocation criteria under task performance: The gendered preference for protection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 45(C), pages 96-111.
    3. Masclet, David & Peterle, Emmanuel & Larribeau, Sophie, 2015. "Gender differences in tournament and flat-wage schemes: An experimental study," Journal of Economic Psychology, Elsevier, vol. 47(C), pages 103-115.
    4. Marchenko, Maria & Sonnabend, Hendrik, 2022. "The Never Ending Book: The role of external stimuli and peer feedback in user-generated content production," Department of Economics Working Paper Series 320, WU Vienna University of Economics and Business.
    5. Dandan Qiao & Shun-Yang Lee & Andrew B. Whinston & Qiang Wei, 2020. "Financial Incentives Dampen Altruism in Online Prosocial Contributions: A Study of Online Reviews," Information Systems Research, INFORMS, vol. 31(4), pages 1361-1375, December.
    6. Jeworrek, Sabrina, 2016. "Competition Entry and Relative Performance Feedback: The Importance of Information Disaggregated by Gender," VfS Annual Conference 2016 (Augsburg): Demographic Change 145859, Verein für Socialpolitik / German Economic Association.
    7. Zahra Murad & Charitini Stavropoulou & Graham Cookson, 2019. "Incentives and gender in a multi-task setting: An experimental study with real-effort tasks," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-18, March.
    8. Muriel Niederle, 2014. "Gender," NBER Working Papers 20788, National Bureau of Economic Research, Inc.
    9. Jetter, Michael & Walker, Jay K., 2018. "The gender of opponents: Explaining gender differences in performance and risk-taking?," European Economic Review, Elsevier, vol. 109(C), pages 238-256.
    10. Ni Huang & Yili Hong & Gordon Burtch, 2015. "Digital Social Visibility, Anonymity and User Content Generation: Evidence from Natural Experiments," Working Papers 15-04, NET Institute.
    11. Yipu Deng & Jinyang Zheng & Warut Khern-am-nuai & Karthik Kannan, 2022. "More than the Quantity: The Value of Editorial Reviews for a User-Generated Content Platform," Management Science, INFORMS, vol. 68(9), pages 6865-6888, September.
    12. De Paola, Maria & Gioia, Francesca & Scoppa, Vincenzo, 2015. "Are females scared of competing with males? Results from a field experiment," Economics of Education Review, Elsevier, vol. 48(C), pages 117-128.
    13. Johnsen, Åshild A. & Finseraas, Henning & Hanson, Torbjørn & Kotsadam, Andreas, 2023. "The malleability of competitive preferences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 104(C).
    14. Dato, Simon & Nieken, Petra, 2014. "Gender differences in competition and sabotage," Journal of Economic Behavior & Organization, Elsevier, vol. 100(C), pages 64-80.
    15. Azmat, Ghazala & Petrongolo, Barbara, 2014. "Gender and the labor market: What have we learned from field and lab experiments?," Labour Economics, Elsevier, vol. 30(C), pages 32-40.
    16. Balafoutas, Loukas & Kerschbamer, Rudolf & Sutter, Matthias, 2012. "Distributional preferences and competitive behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 83(1), pages 125-135.
    17. Jordi Brandts & Valeska Groenert & Christina Rott, 2015. "The Impact of Advice on Women's and Men's Selection into Competition," Management Science, INFORMS, vol. 61(5), pages 1018-1035, May.
    18. Czibor, Eszter & Onderstal, Sander & Sloof, Randolph & van Praag, C. Mirjam, 2020. "Does relative grading help male students? Evidence from a field experiment in the classroom," Economics of Education Review, Elsevier, vol. 75(C).
    19. Josse Delfgaauw & Robert Dur & Joeri Sol & Willem Verbeke, 2013. "Tournament Incentives in the Field: Gender Differences in the Workplace," Journal of Labor Economics, University of Chicago Press, vol. 31(2), pages 305-326.
    20. Jungwon Min, 2022. "Effects of Mixed-Gender Competition: Choking under Pressure in a Dynamic Tournament," IJERPH, MDPI, vol. 19(8), pages 1-14, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:65:y:2019:i:1:p:327-345. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.