IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v68y2022i2p1297-1329.html
   My bibliography  Save this article

Measuring the Impact of Crowdsourcing Features on Mobile App User Engagement and Retention: A Randomized Field Experiment

Author

Listed:
  • Zhuojun Gu

    (Information Systems and Operations Management, The University of Texas at Arlington, Arlington, Texas 76010)

  • Ravi Bapna

    (Information and Decision Sciences, University of Minnesota, Minneapolis, Minnesota 55455)

  • Jason Chan

    (Information and Decision Sciences, University of Minnesota, Minneapolis, Minnesota 55455)

  • Alok Gupta

    (Information and Decision Sciences, University of Minnesota, Minneapolis, Minnesota 55455)

Abstract

The most commonly cited issues with mobile apps are low user engagement and retention levels. In this paper, we use a randomized control trial to test the efficacy of crowdsourcing on enhancing user engagement and retention in the context of mobile gaming apps. We examine two specific crowdsourcing features: the ability to submit content and the ability to access crowdsourced content. We find that the content submission significantly increases engagement and retention by reducing users’ hazard of ending a session by approximately 11% relative to the baseline and reducing the hazard of abandoning the game app by 14%. In contrast, giving users the ability to access crowdsourced content has no significant effect on engagement but has a positive effect on retention by reducing the hazard of abandoning the game app by 13%. Surprisingly, we find that the interaction effect of these two crowdsourced features is negative on user engagement. Individually, the submission feature manifests itself via empowering users to control their product use experience, whereas the access feature’s positive effect on retention is mediated by diversity and novelty of content. However, the two effects are not complementary. It turns out when these two features are given together, the empowerment enabled by one’s own submission is crowded out by others’ submissions, and this dominates the diversity benefit. Crowdsourcing features have heterogeneous impact on different user segments, with heavy users and users of longer tenure being more affected by the crowdsourcing features.

Suggested Citation

  • Zhuojun Gu & Ravi Bapna & Jason Chan & Alok Gupta, 2022. "Measuring the Impact of Crowdsourcing Features on Mobile App User Engagement and Retention: A Randomized Field Experiment," Management Science, INFORMS, vol. 68(2), pages 1297-1329, February.
    • Handle: RePEc:inm:ormnsc:v:68:y:2022:i:2:p:1297-1329
    DOI: 10.1287/mnsc.2020.3943
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.2020.3943
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.2020.3943?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sam Ransbotham & Gerald C. Kane & Nicholas H. Lurie, 2012. "Network Characteristics and the Value of Collaborative User-Generated Content," Marketing Science, INFORMS, vol. 31(3), pages 387-405, May.
    2. Pei-Yu (Sharon) Chen & Lorin M. Hitt, 2002. "Measuring Switching Costs and the Determinants of Customer Retention in Internet-Enabled Businesses: A Study of the Online Brokerage Industry," Information Systems Research, INFORMS, vol. 13(3), pages 255-274, September.
    3. Raghuram Iyengar & Kamel Jedidi & Skander Essegaier & Peter J. Danaher, 2011. "The Impact of Tariff Structure on Customer Retention, Usage, and Profitability of Access Services," Marketing Science, INFORMS, vol. 30(5), pages 820-836, September.
    4. Heather L. O'Brien & Elaine G. Toms, 2010. "The development and evaluation of a survey to measure user engagement," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 50-69, January.
    5. Philipp Afèche & Mojtaba Araghi & Opher Baron, 2017. "Customer Acquisition, Retention, and Service Access Quality: Optimal Advertising, Capacity Level, and Capacity Allocation," Manufacturing & Service Operations Management, INFORMS, vol. 19(4), pages 674-691, October.
    6. Heather L. O'Brien & Elaine G. Toms, 2008. "What is user engagement? A conceptual framework for defining user engagement with technology," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(6), pages 938-955, April.
    7. Anindya Ghose & Sang Pil Han, 2014. "Estimating Demand for Mobile Applications in the New Economy," Management Science, INFORMS, vol. 60(6), pages 1470-1488, June.
    8. Gordon Burtch & Yili Hong & Ravi Bapna & Vladas Griskevicius, 2018. "Stimulating Online Reviews by Combining Financial Incentives and Social Norms," Management Science, INFORMS, vol. 64(5), pages 2065-2082, May.
    9. Yingjie Zhang & Beibei Li & Xueming Luo & Xiaoyi Wang, 2019. "Personalized Mobile Targeting with User Engagement Stages: Combining a Structural Hidden Markov Model and Field Experiment," Information Systems Research, INFORMS, vol. 30(3), pages 787-804, September.
    10. Shlomo Benartzi & Richard H. Thaler, 2002. "How Much Is Investor Autonomy Worth?," Journal of Finance, American Finance Association, vol. 57(4), pages 1593-1616, August.
    11. Ruth N. Bolton, 1998. "A Dynamic Model of the Duration of the Customer's Relationship with a Continuous Service Provider: The Role of Satisfaction," Marketing Science, INFORMS, vol. 17(1), pages 45-65.
    12. Joel O. Wooten & Karl T. Ulrich, 2017. "Idea Generation and the Role of Feedback: Evidence from Field Experiments with Innovation Tournaments," Production and Operations Management, Production and Operations Management Society, vol. 26(1), pages 80-99, January.
    13. Eva Ascarza & Bruce G. S. Hardie, 2013. "A Joint Model of Usage and Churn in Contractual Settings," Marketing Science, INFORMS, vol. 32(4), pages 570-590, July.
    14. Barry L. Bayus, 2013. "Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community," Management Science, INFORMS, vol. 59(1), pages 226-244, June.
    15. Assael, Henry, 2005. "A Demographic and Psychographic Profile of Heavy Internet Users and Users by Type of Internet Usage," Journal of Advertising Research, Cambridge University Press, vol. 45(1), pages 93-123, March.
    16. Eva Ascarza & Oded Netzer & Bruce G. S. Hardie, 2018. "Some Customers Would Rather Leave Without Saying Goodbye," Marketing Science, INFORMS, vol. 37(1), pages 54-77, January.
    17. Dokyun Lee & Kartik Hosanagar & Harikesh S. Nair, 2018. "Advertising Content and Consumer Engagement on Social Media: Evidence from Facebook," Management Science, INFORMS, vol. 64(11), pages 5105-5131, November.
    18. McAlister, Leigh, 1982. "A Dynamic Attribute Satiation Model of Variety-Seeking Behavior," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 9(2), pages 141-150, September.
    19. Yan Huang & Param Vir Singh & Kannan Srinivasan, 2014. "Crowdsourcing New Product Ideas Under Consumer Learning," Management Science, INFORMS, vol. 60(9), pages 2138-2159, September.
    20. Kevin J. Boudreau & Nicola Lacetera & Karim R. Lakhani, 2011. "Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis," Management Science, INFORMS, vol. 57(5), pages 843-863, May.
    21. Khim-Yong Goh & Cheng-Suang Heng & Zhijie Lin, 2013. "Social Media Brand Community and Consumer Behavior: Quantifying the Relative Impact of User- and Marketer-Generated Content," Information Systems Research, INFORMS, vol. 24(1), pages 88-107, March.
    22. Lars Bo Jeppesen & Karim R. Lakhani, 2010. "Marginality and Problem-Solving Effectiveness in Broadcast Search," Organization Science, INFORMS, vol. 21(5), pages 1016-1033, October.
    23. Seshadri Tirunillai & Gerard J. Tellis, 2012. "Does Chatter Really Matter? Dynamics of User-Generated Content and Stock Performance," Marketing Science, INFORMS, vol. 31(2), pages 198-215, March.
    24. McAlister, Leigh & Pessemier, Edgar, 1982. "Variety Seeking Behavior: An Interdisciplinary Review," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 9(3), pages 311-322, December.
    25. Heather L. O'Brien & Elaine G. Toms, 2010. "The development and evaluation of a survey to measure user engagement," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 50-69, January.
    26. Mollen, Anne & Wilson, Hugh, 2010. "Engagement, telepresence and interactivity in online consumer experience: Reconciling scholastic and managerial perspectives," Journal of Business Research, Elsevier, vol. 63(9-10), pages 919-925, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Enache, Andreea & Friberg, Richard & Wiklander, Magnus, 2023. "Demand for in-app purchases in mobile apps—A difference-in-difference approach," International Journal of Industrial Organization, Elsevier, vol. 88(C).
    2. Chen, Claire Y.T. & Sun, Edward W. & Miao, Wanyu & Lin, Yi-Bing, 2024. "Reconciling business analytics with graphically initialized subspace clustering for optimal nonlinear pricing," European Journal of Operational Research, Elsevier, vol. 312(3), pages 1086-1107.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nirup Menon & Anant Mishra & Shun Ye, 2020. "Beyond Related Experience: Upstream vs. Downstream Experience in Innovation Contest Platforms with Interdependent Problem Domains," Manufacturing & Service Operations Management, INFORMS, vol. 22(5), pages 1045-1065, September.
    2. Pallab Sanyal & Shun Ye, 2024. "An Examination of the Dynamics of Crowdsourcing Contests: Role of Feedback Type," Information Systems Research, INFORMS, vol. 35(1), pages 394-413, March.
    3. Ho Cheung Brian Lee & Sulin Ba & Xinxin Li & Jan Stallaert, 2018. "Salience Bias in Crowdsourcing Contests," Information Systems Research, INFORMS, vol. 29(2), pages 401-418, June.
    4. Yuan Jin & Ho Cheung Brian Lee & Sulin Ba & Jan Stallaert, 2021. "Winning by Learning? Effect of Knowledge Sharing in Crowdsourcing Contests," Information Systems Research, INFORMS, vol. 32(3), pages 836-859, September.
    5. repec:eee:respol:v:48:y:2019:i:8:p:- is not listed on IDEAS
    6. Carolina Herrando & Julio Jiménez-Martínez & María José Martín-De Hoyos, 2017. "Passion at first sight: how to engage users in social commerce contexts," Electronic Commerce Research, Springer, vol. 17(4), pages 701-720, December.
    7. Patel, Chirag & Ahmad Husairi, Mariyani & Haon, Christophe & Oberoi, Poonam, 2023. "Monetary rewards and self-selection in design crowdsourcing contests: Managing participation, contribution appropriateness, and winning trade-offs," Technological Forecasting and Social Change, Elsevier, vol. 191(C).
    8. Elina H. Hwang & Param Vir Singh & Linda Argote, 2019. "Jack of All, Master of Some: Information Network and Innovation in Crowdsourcing Communities," Information Systems Research, INFORMS, vol. 30(2), pages 389-410, June.
    9. Jiao, Yuanyuan & Wu, Yepeng & Lu, Steven, 2021. "The role of crowdsourcing in product design: The moderating effect of user expertise and network connectivity," Technology in Society, Elsevier, vol. 64(C).
    10. Swanand J. Deodhar & Samrat Gupta, 2023. "The Impact of Social Reputation Features in Innovation Tournaments: Evidence from a Natural Experiment," Information Systems Research, INFORMS, vol. 34(1), pages 178-193, March.
    11. Bertschek, Irene & Kesler, Reinhold, 2022. "Let the user speak: Is feedback on Facebook a source of firms’ innovation?," Information Economics and Policy, Elsevier, vol. 60(C).
    12. Lars Hornuf & Sabrina Jeworrek, 2018. "How Community Managers Affect Online Idea Crowdsourcing Activities," CESifo Working Paper Series 7153, CESifo.
    13. Christof Naumzik & Stefan Feuerriegel & Markus Weinmann, 2022. "I Will Survive: Predicting Business Failures from Customer Ratings," Marketing Science, INFORMS, vol. 41(1), pages 188-207, January.
    14. Christoph Riedl & Victor P. Seidel, 2018. "Learning from Mixed Signals in Online Innovation Communities," Organization Science, INFORMS, vol. 29(6), pages 1010-1032, December.
    15. Hornuf, Lars & Jeworrek, Sabrina, 2018. "Crowdsourced innovation: How community managers affect crowd activities," IWH Discussion Papers 13/2018, Halle Institute for Economic Research (IWH).
    16. Hu, Feng & Bijmolt, Tammo H.A. & Huizingh, Eelko K.R.E., 2020. "The impact of innovation contest briefs on the quality of solvers and solutions," Technovation, Elsevier, vol. 90.
    17. Vipul Aggarwal & Elina H. Hwang & Yong Tan, 2021. "Learning to Be Creative: A Mutually Exciting Spatiotemporal Point Process Model for Idea Generation in Open Innovation," Information Systems Research, INFORMS, vol. 32(4), pages 1214-1235, December.
    18. Natalicchio, A. & Messeni Petruzzelli, A. & Garavelli, A.C., 2017. "Innovation problems and search for solutions in crowdsourcing platforms – A simulation approach," Technovation, Elsevier, vol. 64, pages 28-42.
    19. Sulin Ba & Barrie R. Nault, 2017. "Emergent Themes in the Interface Between Economics of Information Systems and Management of Technology," Production and Operations Management, Production and Operations Management Society, vol. 26(4), pages 652-666, April.
    20. Dahlander, Linus & Beretta, Michela & Thomas, Arne & Kazemi, Shahab & Fenger, Morten H.J. & Frederiksen, Lars, 2023. "Weeding out or picking winners in open innovation? Factors driving multi-stage crowd selection on LEGO ideas," Research Policy, Elsevier, vol. 52(10).
    21. Gillier, Thomas & Chaffois, Cédric & Belkhouja, Mustapha & Roth, Yannig & Bayus, Barry L., 2018. "The effects of task instructions in crowdsourcing innovative ideas," Technological Forecasting and Social Change, Elsevier, vol. 134(C), pages 35-44.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:68:y:2022:i:2:p:1297-1329. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.