IDEAS home Printed from https://ideas.repec.org/a/spr/decisn/v50y2023i2d10.1007_s40622-023-00349-w.html
   My bibliography  Save this article

Crowd-sourced idea filtering with Bag of Lemons: the impact of the token budget size

Author

Listed:
  • Gafari Lukumon

    (University Mohammed VI Polytechnic
    Ecole Normale Supérieure (ENS-PSL))

  • Mark Klein

    (University Mohammed VI Polytechnic
    Massachusetts Institute of Technology)

Abstract

Identifying the best ideas from the vast volumes generated by open innovation engagements is costly and often time-consuming. One approach is to engage crowds in filtering the ideas, not just generating them. Klein and Garcia, 2015 proposed a “BOL” approach that is better (in terms of accuracy and speed) at idea filtering than other filtering methods such as a conventional Likert approach. The idea behind this approach (BOL) is that it asks the crowd to distribute a fixed budget of tokens that eliminate bad ideas rather than select good ones. In this paper, we explain why BOL works better than other filtering methods using empirical experiments (with n = 850 subjects). Also, we present the effect of the token budget size on idea-filtering engagement and found, among others, that the accuracy of a filter depends on the token budget size.

Suggested Citation

  • Gafari Lukumon & Mark Klein, 2023. "Crowd-sourced idea filtering with Bag of Lemons: the impact of the token budget size," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 50(2), pages 205-219, June.
    • Handle: RePEc:spr:decisn:v:50:y:2023:i:2:d:10.1007_s40622-023-00349-w
    DOI: 10.1007/s40622-023-00349-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40622-023-00349-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40622-023-00349-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Einhorn, Hj & Hogarth, Rm, 1981. "Behavioral Decision-Theory - Processes Of Judgment And Choice," Journal of Accounting Research, Wiley Blackwell, vol. 19(1), pages 1-31.
    2. Eric J. Johnson & John W. Payne, 1985. "Effort and Accuracy in Choice," Management Science, INFORMS, vol. 31(4), pages 395-414, April.
    3. Karim R. Lakhani & Jill A. Panetta, 2007. "The Principles of Distributed Innovation," Innovations: Technology, Governance, Globalization, MIT Press, vol. 2(3), pages 97-112, July.
    4. Joel West & Karim Lakhani, 2008. "Getting Clear About Communities in Open Innovation," Industry and Innovation, Taylor & Francis Journals, vol. 15(2), pages 223-231.
    5. Matthew J Salganik & Karen E C Levy, 2015. "Wiki Surveys: Open and Quantifiable Social Data Collection," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-17, May.
    6. Laura J. Kornish & Jeremy Hutchison‐Krupat, 2017. "Research on Idea Generation and Selection: Implications for Management of Technology," Production and Operations Management, Production and Operations Management Society, vol. 26(4), pages 633-651, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hauser, John R., 2014. "Consideration-set heuristics," Journal of Business Research, Elsevier, vol. 67(8), pages 1688-1699.
    2. Merle, Aurélie & St-Onge, Anik & Sénécal, Sylvain, 2022. "Does it pay to be honest? The effect of retailer-provided negative feedback on consumers’ product choice and shopping experience," Journal of Business Research, Elsevier, vol. 147(C), pages 532-543.
    3. Lior Fink & Daniele Papismedov, 2023. "On the Same Page? What Users Benefit from a Desktop View on Mobile Devices," Information Systems Research, INFORMS, vol. 34(2), pages 423-441, June.
    4. S. Iglesias-Parro & A. Ortega & E. De la Fuente & I. Martín, 2001. "Context Variables as Cognitive Effort Modulators in Decision Making Using an Alternative-Based Processing Strategy," Quality & Quantity: International Journal of Methodology, Springer, vol. 35(3), pages 311-323, August.
    5. O. Alan Tidwell & Paul Gallimore, 2014. "The influence of a decision support tool on real estate valuations," Journal of Property Research, Taylor & Francis Journals, vol. 31(1), pages 45-63, March.
    6. Julia Bauer & Nikolaus Franke & Philipp Tuertscher, 2016. "Intellectual Property Norms in Online Communities: How User-Organized Intellectual Property Regulation Supports Innovation," Information Systems Research, INFORMS, vol. 27(4), pages 724-750, December.
    7. Rullani, Francesco & Haefliger, Stefan, 2013. "The periphery on stage: The intra-organizational dynamics in online communities of creation," Research Policy, Elsevier, vol. 42(4), pages 941-953.
    8. Fischer, Ilan & Budescu, David V., 2005. "When do those who know more also know more about how much they know? The development of confidence and performance in categorical decision tasks," Organizational Behavior and Human Decision Processes, Elsevier, vol. 98(1), pages 39-53, September.
    9. Mingyue Li & Jingjing Wang & Kai Chen & Lianbei Wu, 2020. "Willingness and Behaviors of Farmers’ Green Disposal of Pesticide Packaging Waste in Henan, China: A Perceived Value Formation Mechanism Perspective," IJERPH, MDPI, vol. 17(11), pages 1-18, May.
    10. Gerd Gigerenzer, 1997. "Bounded Rationality: Models of Fast and Frugal Inference," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 133(II), pages 201-218, June.
    11. Nitin Walia & Mark Srite & Wendy Huddleston, 2016. "Eyeing the web interface: the influence of price, product, and personal involvement," Electronic Commerce Research, Springer, vol. 16(3), pages 297-333, September.
    12. Zouaghi, Ferdaous & Sánchez, Mercedes & Martínez, Marian García, 2018. "Did the global financial crisis impact firms' innovation performance? The role of internal and external knowledge capabilities in high and low tech industries," Technological Forecasting and Social Change, Elsevier, vol. 132(C), pages 92-104.
    13. Gauguier, Jean-Jacques, 2009. "L’industrialisation de l’Open Source," Economics Thesis from University Paris Dauphine, Paris Dauphine University, number 123456789/4388 edited by Toledano, Joëlle.
    14. Chorus, Caspar & van Cranenburgh, Sander & Daniel, Aemiro Melkamu & Sandorf, Erlend Dancke & Sobhani, Anae & Szép, Teodóra, 2021. "Obfuscation maximization-based decision-making: Theory, methodology and first empirical evidence," Mathematical Social Sciences, Elsevier, vol. 109(C), pages 28-44.
    15. Patrick Spieth & Svenja Meissner Née Schuchert, 2018. "Business Model Innovation Alliances: How To Open Business Models For Cooperation," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 22(04), pages 1-26, May.
    16. Dolata, Ulrich, 2014. "Märkte und Macht der Internetkonzerne: Konzentration - Konkurrenz - Innovationsstrategien," Research Contributions to Organizational Sociology and Innovation Studies, SOI Discussion Papers 2014-04, University of Stuttgart, Institute for Social Sciences, Department of Organizational Sociology and Innovation Studies.
    17. S. Larsson & G. R. Chesley, 1986. "An analysis of the auditor's uncertainty about probabilities," Contemporary Accounting Research, John Wiley & Sons, vol. 2(2), pages 259-282, March.
    18. Yu‐Lin Hsu & Gavin C. Reid, 2021. "Two‐stage decision‐making within the firm: Analysis and case studies," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(6), pages 1355-1373, September.
    19. Gove Allen & Jeffrey Parsons, 2010. "Is Query Reuse Potentially Harmful? Anchoring and Adjustment in Adapting Existing Database Queries," Information Systems Research, INFORMS, vol. 21(1), pages 56-77, March.
    20. Karen Green & Benson Wier, 2015. "Influence of Ethical Position and Information Asymmetry on Transfer Price Negotiations," Accounting and Finance Research, Sciedu Press, vol. 4(1), pages 1-30, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:decisn:v:50:y:2023:i:2:d:10.1007_s40622-023-00349-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.