IDEAS home Printed from https://ideas.repec.org/a/inm/ormksc/v43y2024i1p54-72.html
   My bibliography  Save this article

Can AI Help in Ideation? A Theory-Based Model for Idea Screening in Crowdsourcing Contests

Author

Listed:
  • J. Jason Bell

    (Saïd Business School, University of Oxford, Oxford OX1 1HP, United Kingdom)

  • Christian Pescher

    (Universidad de los Andes, Las Condes, Santiago, Chile)

  • Gerard J. Tellis

    (Marshall School of Business, University of Southern California, Los Angeles, California 90089)

  • Johann Füller

    (University of Innsbruck, 6020 Innsbruck, Austria)

Abstract

Crowdsourcing generates up to thousands of ideas per contest. The selection of best ideas is costly because of the limited number, objectivity, and attention of experts. Using a data set of 21 crowdsourcing contests that include 4,191 ideas, we test how artificial intelligence can assist experts in screening ideas. The authors have three major findings. First, whereas even the best previously published theory-based models cannot mimic human experts in choosing the best ideas, a simple model using the least average shrinkage and selection operator can efficiently screen out ideas considered bad by experts. In an additional 22nd hold-out contest with internal and external experts, the simple model does better than external experts in predicting the ideas selected by internal experts. Second, the authors develop an idea screening efficiency curve that trades off the false negative rate against the total ideas screened. Managers can choose the desired point on this curve given their loss function. The best model specification can screen out 44% of ideas, sacrificing only 14% of good ideas. Alternatively, for those unwilling to lose any winners, a novel two-step approach screens out 21% of ideas without sacrificing a single first place winner. Third, a new predictor, word atypicality, is simple and efficient in screening. Theoretically, this predictor screens out atypical ideas and keeps inclusive and rich ideas.

Suggested Citation

  • J. Jason Bell & Christian Pescher & Gerard J. Tellis & Johann Füller, 2024. "Can AI Help in Ideation? A Theory-Based Model for Idea Screening in Crowdsourcing Contests," Marketing Science, INFORMS, vol. 43(1), pages 54-72, January.
    • Handle: RePEc:inm:ormksc:v:43:y:2024:i:1:p:54-72
    DOI: 10.1287/mksc.2023.1434
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mksc.2023.1434
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mksc.2023.1434?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Laura J. Kornish & Sharaya M. Jones, 2021. "Raw Ideas in the Fuzzy Front End: Verbosity Increases Perceived Creativity," Marketing Science, INFORMS, vol. 40(6), pages 1106-1122, November.
    2. King, Gary & Zeng, Langche, 2001. "Logistic Regression in Rare Events Data," Political Analysis, Cambridge University Press, vol. 9(2), pages 137-163, January.
    3. Goldenberg,Jacob & Mazursky,David, 2002. "Creativity in Product Innovation," Cambridge Books, Cambridge University Press, number 9780521800891, October.
    4. Goldenberg,Jacob & Mazursky,David, 2002. "Creativity in Product Innovation," Cambridge Books, Cambridge University Press, number 9780521002493, October.
    5. Oded Netzer & Ronen Feldman & Jacob Goldenberg & Moshe Fresko, 2012. "Mine Your Own Business: Market-Structure Surveillance Through Text Mining," Marketing Science, INFORMS, vol. 31(3), pages 521-543, May.
    6. Omid Rafieian & Hema Yoganarasimhan, 2021. "Targeting and Privacy in Mobile Advertising," Marketing Science, INFORMS, vol. 40(2), pages 193-218, March.
    7. Olivier Toubia, 2006. "Idea Generation, Creativity, and Incentives," Marketing Science, INFORMS, vol. 25(5), pages 411-425, September.
    8. Laura J. Kornish & Karl T. Ulrich, 2011. "Opportunity Spaces in Innovation: Empirical Analysis of Large Samples of Ideas," Management Science, INFORMS, vol. 57(1), pages 107-128, January.
    9. Olivier Toubia & Laurent Florès, 2007. "Adaptive Idea Screening Using Consumers," Marketing Science, INFORMS, vol. 26(3), pages 342-360, 05-06.
    10. Olivier Toubia & Oded Netzer, 2017. "Idea Generation, Creativity, and Prototypicality," Marketing Science, INFORMS, vol. 36(1), pages 1-20, January.
    11. Frédéric C. Godart & Charles Galunic, 2019. "Explaining the Popularity of Cultural Elements: Networks, Culture, and the Structural Embeddedness of High Fashion Trends," Organization Science, INFORMS, vol. 30(1), pages 151-168, February.
    12. Barry L. Bayus, 2013. "Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community," Management Science, INFORMS, vol. 59(1), pages 226-244, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Olivier Toubia & Oded Netzer, 2017. "Idea Generation, Creativity, and Prototypicality," Marketing Science, INFORMS, vol. 36(1), pages 1-20, January.
    2. Niek Althuizen & Bo Chen, 2022. "Crowdsourcing Ideas Using Product Prototypes: The Joint Effect of Prototype Enhancement and the Product Design Goal on Idea Novelty," Management Science, INFORMS, vol. 68(4), pages 3008-3025, April.
    3. Laura J. Kornish & Jeremy Hutchison‐Krupat, 2017. "Research on Idea Generation and Selection: Implications for Management of Technology," Production and Operations Management, Production and Operations Management Society, vol. 26(4), pages 633-651, April.
    4. Dominik Mahr & Aric Rindfleisch & Rebecca Slotegraaf, 2015. "Enhancing Crowdsourcing Success: the Role of Creative and Deliberate Problem-Solving Styles," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 2(3), pages 209-221, September.
    5. Juncai Jiang & Yu Wang, 2020. "A Theoretical and Empirical Investigation of Feedback in Ideation Contests," Production and Operations Management, Production and Operations Management Society, vol. 29(2), pages 481-500, February.
    6. Douglas Cumming & Lars Hornuf & Moein Karami & Denis Schweizer, 2023. "Disentangling Crowdfunding from Fraudfunding," Journal of Business Ethics, Springer, vol. 182(4), pages 1103-1128, February.
    7. Ellen H.M. Moors & Rogier Donders, 2009. "Understanding consumer needs and preferences in new product development: the case of functional food innovations," Innovation Studies Utrecht (ISU) working paper series 09-03, Utrecht University, Department of Innovation Studies, revised Feb 2009.
    8. Hassan Arabshahi & Hamed Fazlollahtabar, 2018. "Classifying Innovative Activities Using Decision Tree and Gini Index," International Journal of Innovation and Technology Management (IJITM), World Scientific Publishing Co. Pte. Ltd., vol. 15(03), pages 1-14, June.
    9. Barry L. Bayus, 2013. "Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community," Management Science, INFORMS, vol. 59(1), pages 226-244, June.
    10. Wang Kai & Tao Yu & Wang Hui, 2017. "Combining Ideas in Crowdsourced Idea Generation," Foundations of Management, Sciendo, vol. 9(1), pages 203-212, February.
    11. Laura J. Kornish & Sharaya M. Jones, 2021. "Raw Ideas in the Fuzzy Front End: Verbosity Increases Perceived Creativity," Marketing Science, INFORMS, vol. 40(6), pages 1106-1122, November.
    12. repec:wsi:acsxxx:v:21:y:2019:i:08:n:s1363919619500142 is not listed on IDEAS
    13. Liao, Junyun & Chen, Jiawen & Mou, Jian, 2021. "Examining the antecedents of idea contribution in online innovation communities: A perspective of creative self-efficacy," Technology in Society, Elsevier, vol. 66(C).
    14. Oguz Ali Acar, 2018. "Harnessing the creative potential of consumers: money, participation, and creativity in idea crowdsourcing," Marketing Letters, Springer, vol. 29(2), pages 177-188, June.
    15. Thomas Görzen & Dennis Kundisch, 2019. "When in Doubt Follow the Crowd: How Idea Quality Moderates the Effect of an Anchor on Idea Evaluation," Working Papers Dissertations 57, Paderborn University, Faculty of Business Administration and Economics.
    16. Boons, Mark & Stam, Daan, 2019. "Crowdsourcing for innovation: How related and unrelated perspectives interact to increase creative performance," Research Policy, Elsevier, vol. 48(7), pages 1758-1770.
    17. Marc R. Dotson & Joachim Büschken & Greg M. Allenby, 2020. "Explaining Preference Heterogeneity with Mixed Membership Modeling," Marketing Science, INFORMS, vol. 39(2), pages 407-426, March.
    18. Bicen, Pelin & Kamarudin, Suzilawati & Johnson, William H.A., 2014. "Validating new product creativity in the eastern context of Malaysia," Journal of Business Research, Elsevier, vol. 67(1), pages 2877-2883.
    19. Feng, Yuanyue & Yi, Zihui & Yang, Congcong & Chen, Ruoyi & Feng, Ye, 2022. "How do gamification mechanics drive solvers’ Knowledge contribution? A study of collaborative knowledge crowdsourcing," Technological Forecasting and Social Change, Elsevier, vol. 177(C).
    20. Gaetano Miceli & Maria Antonietta Raimondo, 2020. "Creativity in the marketing and consumer behavior literature: a structured review and a research agenda," Italian Journal of Marketing, Springer, vol. 2020(1), pages 85-124, March.
    21. Nikolaus Franke & Peter Keinz & Katharina Klausberger, 2013. "“Does This Sound Like a Fair Deal?”: Antecedents and Consequences of Fairness Expectations in the Individual’s Decision to Participate in Firm Innovation," Organization Science, INFORMS, vol. 24(5), pages 1495-1516, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormksc:v:43:y:2024:i:1:p:54-72. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.