The promise of pick-the-winners contests for producing crowd probability forecasts

This paper considers pick-the-winners contests as a simple method for harnessing the wisdom of crowds to produce probability forecasts (and be used as a business forecasting tool). Pick-the-winners contests are those in which players pick the outcomes of selected future binary events with a prize going to the player with the most correct picks. In contrast to soliciting probability forecasts from experts (for which competition among the forecasters leads to exaggerated and less accurate probabilities), this paper shows that competition among players is to be encouraged because it improves the accuracy of the resulting crowd probability forecasts. This improvement comes because the competition not only discourages the overbetting of favorites that occurs if prizes are awarded based on absolute performance, but also helps mitigate the public knowledge bias that occurs if everyone reports truthfully. In addition to the theoretical arguments, the paper analyzes picks from 6 years of pick-the-winners contests involving hundreds of players and 1037 college lacrosse games to find that the crowd proportions outperformed two benchmark sets of probabilities.