IDEAS home Printed from https://ideas.repec.org/a/taf/amstat/v68y2014i4p264-270.html
   My bibliography  Save this article

The Promise of Prediction Contests

Author

Listed:
  • Phillip E. Pfeifer
  • Yael Grushka-Cockayne
  • Kenneth C. Lichtendahl

Abstract

This article examines the prediction contest as a vehicle for aggregating the opinions of a crowd of experts. After proposing a general definition distinguishing prediction contests from other mechanisms for harnessing the wisdom of crowds, we focus on point-forecasting contests-contests in which forecasters submit point forecasts with a prize going to the entry closest to the quantity of interest. We first illustrate the incentive for forecasters to submit reports that exaggerate in the direction of their private information. Whereas this exaggeration raises a forecaster's mean squared error, it increases his or her chances of winning the contest. And in contrast to conventional wisdom, this nontruthful reporting usually improves the accuracy of the resulting crowd forecast. The source of this improvement is that exaggeration shifts weight away from public information (information known to all forecasters) and by so doing helps alleviate public knowledge bias. In the context of a simple theoretical model of overlapping information and forecaster behaviors, we present closed-form expressions for the mean squared error of the crowd forecasts which will help identify the situations in which point forecasting contests will be most useful.

Suggested Citation

  • Phillip E. Pfeifer & Yael Grushka-Cockayne & Kenneth C. Lichtendahl, 2014. "The Promise of Prediction Contests," The American Statistician, Taylor & Francis Journals, vol. 68(4), pages 264-270, November.
  • Handle: RePEc:taf:amstat:v:68:y:2014:i:4:p:264-270
    DOI: 10.1080/00031305.2014.937545
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/00031305.2014.937545
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1080/00031305.2014.937545?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
    3. Bryan Clair & David Letscher, 2007. "Optimal Strategies for Sports Betting Pools," Operations Research, INFORMS, vol. 55(6), pages 1163-1177, December.
    4. Oliver Kim & Steve C. Lim & Kenneth W. Shaw, 2001. "The Inefficiency of the Mean Analyst Forecast as a Summary Forecast of Earnings," Journal of Accounting Research, Wiley Blackwell, vol. 39(2), pages 329-335, September.
    5. Kay-Yut Chen & Leslie R. Fine & Bernardo A. Huberman, 2004. "Eliminating Public Knowledge Biases in Information-Aggregation Mechanisms," Management Science, INFORMS, vol. 50(7), pages 983-994, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sepideh Bazazi & Jorina von Zimmermann & Bahador Bahrami & Daniel Richardson, 2019. "Self-serving incentives impair collective decisions by increasing conformity," PLOS ONE, Public Library of Science, vol. 14(11), pages 1-12, November.
    2. Ronald Peeters & Fan Rao & Leonard Wolk, 2022. "Small group forecasting using proportional-prize contests," Theory and Decision, Springer, vol. 92(2), pages 293-317, March.
    3. Brown, Alasdair & Reade, J. James, 2019. "The wisdom of amateur crowds: Evidence from an online community of sports tipsters," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1073-1081.
    4. Yanwei Jia & Jussi Keppo & Ville Satopää, 2023. "Herding in Probabilistic Forecasts," Management Science, INFORMS, vol. 69(5), pages 2713-2732, May.
    5. Phillip E. Pfeifer, 2016. "The promise of pick-the-winners contests for producing crowd probability forecasts," Theory and Decision, Springer, vol. 81(2), pages 255-278, August.
    6. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
    7. Keppo, Jussi & Satopää, Ville A., 2024. "Bayesian herd detection for dynamic data," International Journal of Forecasting, Elsevier, vol. 40(1), pages 285-301.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Phillip E. Pfeifer, 2016. "The promise of pick-the-winners contests for producing crowd probability forecasts," Theory and Decision, Springer, vol. 81(2), pages 255-278, August.
    2. Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Phillip E. Pfeifer, 2013. "The Wisdom of Competitive Crowds," Operations Research, INFORMS, vol. 61(6), pages 1383-1398, December.
    3. Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
    4. Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
    5. Keppo, Jussi & Satopää, Ville A., 2024. "Bayesian herd detection for dynamic data," International Journal of Forecasting, Elsevier, vol. 40(1), pages 285-301.
    6. Marinovic, Iván & Ottaviani, Marco & Sorensen, Peter, 2013. "Forecasters’ Objectives and Strategies," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 690-720, Elsevier.
    7. Reslow, André, 2019. "Inefficient Use of Competitors'Forecasts?," Working Paper Series 380, Sveriges Riksbank (Central Bank of Sweden).
    8. Yael Grushka-Cockayne & Victor Richmond R. Jose & Kenneth C. Lichtendahl Jr., 2017. "Ensembles of Overfit and Overconfident Forecasts," Management Science, INFORMS, vol. 63(4), pages 1110-1130, April.
    9. Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
    10. Stekler Herman O. & Klein Andrew, 2012. "Predicting the Outcomes of NCAA Basketball Championship Games," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 8(1), pages 1-10, March.
    11. Berna Karali & Scott H. Irwin & Olga Isengildina‐Massa, 2020. "Supply Fundamentals and Grain Futures Price Movements," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(2), pages 548-568, March.
    12. Aleksei Smirnov & Egor Starkov, 2019. "Timing of predictions in dynamic cheap talk: experts vs. quacks," ECON - Working Papers 334, Department of Economics - University of Zurich.
    13. Crowe, Christopher & Meade, Ellen E., 2008. "Central bank independence and transparency: Evolution and effectiveness," European Journal of Political Economy, Elsevier, vol. 24(4), pages 763-777, December.
    14. Cukierman, Alex & Lustenberger, Thomas, 2017. "International evidence on professional interest rates forecasts: The impact of forecasting ability," CEPR Discussion Papers 12489, C.E.P.R. Discussion Papers.
    15. Rybacki Jakub, 2020. "Macroeconomic forecasting in Poland: The role of forecasting competitions," Central European Economic Journal, Sciendo, vol. 7(54), pages 1-11, January.
    16. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    17. Fritsche, Ulrich & Pierdzioch, Christian & Rülke, Jan-Christoph & Stadtmann, Georg, 2015. "Forecasting the Brazilian real and the Mexican peso: Asymmetric loss, forecast rationality, and forecaster herding," International Journal of Forecasting, Elsevier, vol. 31(1), pages 130-139.
    18. Bespalova, Olga, 2018. "Forecast Evaluation in Macroeconomics and International Finance. Ph.D. thesis, George Washington University, Washington, DC, USA," MPRA Paper 117706, University Library of Munich, Germany.
    19. repec:cup:judgdm:v:15:y:2020:i:5:p:863-880 is not listed on IDEAS
    20. David Bergman & Carlos Cardonha & Jason Imbrogno & Leonardo Lozano, 2023. "Optimizing the Expected Maximum of Two Linear Functions Defined on a Multivariate Gaussian Distribution," INFORMS Journal on Computing, INFORMS, vol. 35(2), pages 304-317, March.
    21. Jordi Blanes, 2003. "Credibility and Cheap Talk of Securities Analysts:Theory and Evidence," FMG Discussion Papers dp472, Financial Markets Group.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:amstat:v:68:y:2014:i:4:p:264-270. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UTAS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.