Hidden Experts in the Crowd: Using Meta-Predictions to Leverage Expertise in Single-Question Prediction Problems
Author
Abstract
Suggested Citation
DOI: 10.1287/mnsc.2020.3919
Download full text from publisher
References listed on IDEAS
- David V. Budescu & Eva Chen, 2015. "Identifying Expertise to Extract the Wisdom of Crowds," Management Science, INFORMS, vol. 61(2), pages 267-280, February.
- Genre, Véronique & Kenny, Geoff & Meyler, Aidan & Timmermann, Allan, 2013. "Combining expert forecasts: Can anything beat the simple average?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 108-121.
- Johannes Müller-Trede & Shoham Choshen-Hillel & Meir Barneron & Ilan Yaniv, 2017. "The Wisdom of Crowds in Matters of Taste," Discussion Paper Series dp709, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
- Ville A. Satopää & Robin Pemantle & Lyle H. Ungar, 2016. "Modeling Probability Forecasts via Information Diversity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1623-1633, October.
- Michael D. Lee & Megan N. Lee, 2017. "The relationship between crowd majority and accuracy for binary decisions," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 12(4), pages 328-343, July.
- Dražen Prelec & H. Sebastian Seung & John McCoy, 2017. "A solution to the single-question crowd wisdom problem," Nature, Nature, vol. 541(7638), pages 532-535, January.
- David Court & Benjamin Gillen & Jordi McKenzie & Charles R. Plott, 2018.
"Two information aggregation mechanisms for predicting the opening weekend box office revenues of films: Boxoffice Prophecy and Guess of Guesses,"
Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(1), pages 25-54, January.
- David Court & Benjamin Gillen & Jordi McKenzie & Charles Plott, 2015. "Two Information Aggregation Mechanisms for Predicting the Opening Weekend Box Office Revenues of Films: Boxoffice Prophecy and Guess of Guesses," Natural Field Experiments 00541, The Field Experiments Website.
- Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
- repec:cup:judgdm:v:12:y:2017:i:4:p:328-343 is not listed on IDEAS
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
- Cem Peker, 2023. "Extracting the collective wisdom in probabilistic judgments," Theory and Decision, Springer, vol. 94(3), pages 467-501, April.
- Asa B. Palley & Jack B. Soll, 2019. "Extracting the Wisdom of Crowds When Information Is Shared," Management Science, INFORMS, vol. 67(5), pages 2291-2309, May.
- Satopää, Ville A., 2021. "Improving the wisdom of crowds with analysis of variance of predictions of related outcomes," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1728-1747.
- Satopää, Ville A. & Salikhov, Marat & Tetlock, Philip E. & Mellers, Barbara, 2023. "Decomposing the effects of crowd-wisdom aggregators: The bias–information–noise (BIN) model," International Journal of Forecasting, Elsevier, vol. 39(1), pages 470-485.
- Mauro Costantini & Ulrich Gunter & Robert M. Kunst, 2017.
"Forecast Combinations in a DSGE‐VAR Lab,"
Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(3), pages 305-324, April.
- Costantini, Mauro & Gunter, Ulrich & Kunst, Robert M., 2014. "Forecast combinations in a DSGE-VAR lab," Economics Series 309, Institute for Advanced Studies.
- Benchimol, Jonathan & El-Shagi, Makram & Saadon, Yossi, 2022.
"Do expert experience and characteristics affect inflation forecasts?,"
Journal of Economic Behavior & Organization, Elsevier, vol. 201(C), pages 205-226.
- Jonathan Benchimol & Makram El-Shagi & Yossi Saadon, 2020. "Do Expert Experience and Characteristics Affect Inflation Forecasts?," Bank of Israel Working Papers 2020.11, Bank of Israel.
- Jonathan Benchimol & Makram El-Shagi & Yossi Saadon, 2022. "Do expert experience and characteristics affect inflation forecasts?," Post-Print emse-04624966, HAL.
- Jonathan Benchimol & Makram El-Shagi & Yossi Saadon, 2020. "Do Expert Experience and Characteristics Affect Inflation Forecasts?," CFDS Discussion Paper Series 2020/6, Center for Financial Development and Stability at Henan University, Kaifeng, Henan, China.
- Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015.
"Optimal combination of survey forecasts,"
International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
- Giannone, Domenico & De Mol, Christine & Conflitti, Cristina, 2012. "Optimal Combination of Survey Forecasts," CEPR Discussion Papers 9096, C.E.P.R. Discussion Papers.
- Cristina Conflitti & Christine De Mol & Domenico Giannone, 2012. "Optimal Combination of Survey Forecasts," Working Papers ECARES ECARES 2012-023, ULB -- Universite Libre de Bruxelles.
- Diebold, Francis X. & Shin, Minchul, 2019.
"Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives,"
International Journal of Forecasting, Elsevier, vol. 35(4), pages 1679-1691.
- Francis X. Diebold & Minchul Shin, 2018. "Machine Learning for Regularized Survey Forecast Combination: Partially-Egalitarian Lasso and its Derivatives," NBER Working Papers 24967, National Bureau of Economic Research, Inc.
- Francis X. Diebold & Minchul Shin, 2018. "Machine Learning for Regularized Survey Forecast Combination: Partially Egalitarian Lasso and its Derivatives," PIER Working Paper Archive 18-014, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 17 Aug 2018.
- Sebastian M. Blanc & Thomas Setzer, 2020. "Bias–Variance Trade-Off and Shrinkage of Weights in Forecast Combination," Management Science, INFORMS, vol. 66(12), pages 5720-5737, December.
- Shinitzky, Hilla & Shemesh, Yhonatan & Leiser, David & Gilead, Michael, 2024. "Improving geopolitical forecasts with 100 brains and one computer," International Journal of Forecasting, Elsevier, vol. 40(3), pages 958-970.
- Marcellin Martinie & Tom Wilkening & Piers D L Howe, 2020. "Using meta-predictions to identify experts in the crowd when past performance is unknown," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-11, April.
- Yongchen Zhao, 2021.
"The robustness of forecast combination in unstable environments: a Monte Carlo study of advanced algorithms,"
Empirical Economics, Springer, vol. 61(1), pages 173-199, July.
- Yongchen Zhao, 2015. "Robustness of Forecast Combination in Unstable Environment: A Monte Carlo Study of Advanced Algorithms," Working Papers 2015-005, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
- Yongchen Zhao, 2015. "Robustness of Forecast Combination in Unstable Environment: A Monte Carlo Study of Advanced Algorithms," Working Papers 2015-04, Towson University, Department of Economics, revised Mar 2020.
- Lisheng He & Pantelis P. Analytis & Sudeep Bhatia, 2022. "The Wisdom of Model Crowds," Management Science, INFORMS, vol. 68(5), pages 3635-3659, May.
- Martin, Gael M. & Loaiza-Maya, Rubén & Maneesoonthorn, Worapree & Frazier, David T. & Ramírez-Hassan, Andrés, 2022.
"Optimal probabilistic forecasts: When do they work?,"
International Journal of Forecasting, Elsevier, vol. 38(1), pages 384-406.
- Ruben Loaiza-Maya & Gael M. Martin & David T. Frazier & Worapree Maneesoonthorn & Andres Ramirez Hassan, 2020. "Optimal probabilistic forecasts: When do they work?," Monash Econometrics and Business Statistics Working Papers 33/20, Monash University, Department of Econometrics and Business Statistics.
- Gael M. Martin & Rub'en Loaiza-Maya & David T. Frazier & Worapree Maneesoonthorn & Andr'es Ram'irez Hassan, 2020. "Optimal probabilistic forecasts: When do they work?," Papers 2009.09592, arXiv.org.
- Johannes Müller-Trede & Shoham Choshen-Hillel & Meir Barneron & Ilan Yaniv, 2018. "The Wisdom of Crowds in Matters of Taste," Management Science, INFORMS, vol. 64(4), pages 1779-1803, April.
- Jakub Nowotarski & Rafał Weron, 2015.
"Computing electricity spot price prediction intervals using quantile regression and forecast averaging,"
Computational Statistics, Springer, vol. 30(3), pages 791-803, September.
- Jakub Nowotarski & Rafal Weron, 2013. "Computing electricity spot price prediction intervals using quantile regression and forecast averaging," HSC Research Reports HSC/13/12, Hugo Steinhaus Center, Wroclaw University of Science and Technology.
- Philip Hans Franses, 2024. "Incorporating judgment in forecasting models in times of crisis," Futures & Foresight Science, John Wiley & Sons, vol. 6(4), December.
- Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
- Kang, Yanfei & Cao, Wei & Petropoulos, Fotios & Li, Feng, 2022. "Forecast with forecasts: Diversity matters," European Journal of Operational Research, Elsevier, vol. 301(1), pages 180-190.
More about this item
Keywords
expertise; meta-knowledge; wisdom of crowds; forecasting; aggregation;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:68:y:2022:i:1:p:487-508. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.