When in Doubt Follow the Crowd: How Idea Quality Moderates the Effect of an Anchor on Idea Evaluation

Companies increasingly engage the crowd in the evaluation of a large pool of ideas to sift out the better ones among them. The crowd, however, seems to be better at eliminating the worst ideas than identifying the better ones. Using the anchoring effect as a treatment, and the decreasing effect on the variance of ratings, we develop an approach that enables the use of crowd evaluation for the identification of high-quality ideas. To investigate whether our approach is effective, we conduct several experiments on a crowdworking-platform. Our empirical results indicate that evaluating ideas of high quality represents a more challenging task for the crowd than evaluating those of low quality. Accordingly, idea quality moderates the effect of an anchor for idea evaluation. Following a mixed methods approach, results of an additional qualitative study support our empirical results, indicating that participants are less certain about their evaluation when evaluating ideas of high quality. Our findings both extend the existing literature on crowd evaluation and offer practical solutions for how a crowd can be used to identify the most promising ideas.\\