IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v69y2016i10p4081-4094.html
   My bibliography  Save this article

Testing weighting approaches for forecasting in a Group Wisdom Support System environment

Author

Listed:
  • von der Gracht, Heiko A.
  • Hommel, Ulrich
  • Prokesch, Tobias
  • Wohlenberg, Holger

Abstract

Decision makers usually seek the best possible information to support their decisions. Yet the more experts a decision maker consults, the more divergent opinions he or she might collect. In particular, the approach of attaining an adequate level of information is of crucial importance for many stakeholders such as financial and political institutions as well as sales departments. Inspired by fact that simple heuristics oftentimes outperform complex optimization models, we test and compare several simple forecast-combining methods, including multiple equally weighted approaches, an “imitate-the-successful” heuristic as well as several other weighting approaches (based on self-assessment, knowledge, and hit rate). Forecasts are collected and processed from the crowd using a novel Group Wisdom Support System (GWSS), which provides an entire forecast distribution and information on the consensus evolution over time. We find that the equally weighted triangular forecasts, a simple 1/N heuristic, delivers the most accurate results.

Suggested Citation

  • von der Gracht, Heiko A. & Hommel, Ulrich & Prokesch, Tobias & Wohlenberg, Holger, 2016. "Testing weighting approaches for forecasting in a Group Wisdom Support System environment," Journal of Business Research, Elsevier, vol. 69(10), pages 4081-4094.
  • Handle: RePEc:eee:jbrese:v:69:y:2016:i:10:p:4081-4094
    DOI: 10.1016/j.jbusres.2016.03.043
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0148296316300637
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jbusres.2016.03.043?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    2. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    3. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    4. Snowberg, Erik & Wolfers, Justin & Zitzewitz, Eric, 2013. "Prediction Markets for Economic Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 657-687, Elsevier.
    5. Graefe, Andreas & Armstrong, J. Scott, 2011. "Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task," International Journal of Forecasting, Elsevier, vol. 27(1), pages 183-195, January.
    6. Richard P. Larrick & Jack B. Soll, 2006. "Erratum--Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(2), pages 309-310, February.
    7. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    8. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    9. Lobo, Gerald J., 1991. "Alternative methods of combining security analysts' and statistical forecasts of annual corporate earnings," International Journal of Forecasting, Elsevier, vol. 7(1), pages 57-63, May.
    10. Deaves, Richard & Lüders, Erik & Schröder, Michael, 2010. "The dynamics of overconfidence: Evidence from stock market forecasters," Journal of Economic Behavior & Organization, Elsevier, vol. 75(3), pages 402-412, September.
    11. Schultze, Thomas & Mojzisch, Andreas & Schulz-Hardt, Stefan, 2012. "Why groups perform better than individuals at quantitative judgment tasks: Group-to-individual transfer as an alternative to differential weighting," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 24-36.
    12. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    13. Elena Andreou & Eric Ghysels & Andros Kourtellos, 2013. "Should Macroeconomic Forecasters Use Daily Financial Data and How?," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(2), pages 240-251, April.
    14. Alison Hubbard Ashton & Robert H. Ashton, 1985. "Aggregating Subjective Forecasts: Some Empirical Results," Management Science, INFORMS, vol. 31(12), pages 1499-1508, December.
    15. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 4, pages 135-196, Elsevier.
    16. Francis X. Diebold & Anthony S. Tay & Kenneth F. Wallis, 1997. "Evaluating Density Forecasts of Inflation: The Survey of Professional Forecasters," NBER Working Papers 6228, National Bureau of Economic Research, Inc.
    17. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    18. Capistrán, Carlos & Timmermann, Allan, 2009. "Forecast Combination With Entry and Exit of Experts," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 428-440.
    19. Genre, Véronique & Kenny, Geoff & Meyler, Aidan & Timmermann, Allan, 2013. "Combining expert forecasts: Can anything beat the simple average?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 108-121.
    20. Olsson, Henrik, 2014. "Measuring overconfidence: Methodological problems and statistical artifacts," Journal of Business Research, Elsevier, vol. 67(8), pages 1766-1770.
    21. Robert T. Clemen & Robert L. Winkler, 1985. "Limits for the Precision and Value of Information from Dependent Sources," Operations Research, INFORMS, vol. 33(2), pages 427-442, April.
    22. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    23. Hilde C. Bjørnland & Karsten Gerdrup & Anne Sofie Jore & Christie Smith & Leif Anders Thorsrud, 2012. "Does Forecast Combination Improve Norges Bank Inflation Forecasts?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 74(2), pages 163-179, April.
    24. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    25. Menkhoff, Lukas & Schmeling, Maik & Schmidt, Ulrich, 2013. "Overconfidence, experience, and professionalism: An experimental study," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 92-101.
    26. Balduzzi, Pierluigi & Elton, Edwin J. & Green, T. Clifton, 2001. "Economic News and Bond Prices: Evidence from the U.S. Treasury Market," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 36(4), pages 523-543, December.
    27. Holden, K. & Peel, D. A., 1986. "Expectations formation, public forecasts and the wage equation," Economic Modelling, Elsevier, vol. 3(2), pages 129-134, April.
    28. Lau, Yeng Wai, 2014. "Aggregated or disaggregated information first?," Journal of Business Research, Elsevier, vol. 67(11), pages 2376-2384.
    29. Diebold, Francis X. & Pauly, Peter, 1990. "The use of prior information in forecast combination," International Journal of Forecasting, Elsevier, vol. 6(4), pages 503-508, December.
    30. Sorkin, Robert D. & Dai, Huanping, 1994. "Signal Detection Analysis of the Ideal Group," Organizational Behavior and Human Decision Processes, Elsevier, vol. 60(1), pages 1-13, October.
    31. Graefe, Andreas & Armstrong, J. Scott & Jones, Randall J. & Cuzán, Alfred G., 2014. "Combining forecasts: An application to elections," International Journal of Forecasting, Elsevier, vol. 30(1), pages 43-54.
    32. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    33. Hinz, Oliver & Schulze, Christian & Takac, Carsten, 2014. "New product adoption in social networks: Why direction matters," Journal of Business Research, Elsevier, vol. 67(1), pages 2836-2844.
    34. Dan Lovallo & Colin Camerer, 1999. "Overconfidence and Excess Entry: An Experimental Approach," American Economic Review, American Economic Association, vol. 89(1), pages 306-318, March.
    35. Jose, Victor Richmond R. & Winkler, Robert L., 2008. "Simple robust averages of forecasts: Some empirical results," International Journal of Forecasting, Elsevier, vol. 24(1), pages 163-169.
    36. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    37. Kenneth Wallis, 2011. "Combining forecasts - forty years later," Applied Financial Economics, Taylor & Francis Journals, vol. 21(1-2), pages 33-41.
    38. Menkhoff, Lukas & Schmidt, Ulrich & Brozynski, Torsten, 2006. "The impact of experience on risk taking, overconfidence, and herding of fund managers: Complementary survey evidence," European Economic Review, Elsevier, vol. 50(7), pages 1753-1766, October.
    39. Shanteau, James & Weiss, David J. & Thomas, Rickey P. & Pounds, Julia C., 2002. "Performance-based assessment of expertise: How to decide if someone is an expert or not," European Journal of Operational Research, Elsevier, vol. 136(2), pages 253-263, January.
    40. Oberlechner, Thomas & Osler, Carol, 2012. "Survival of Overconfidence in Currency Markets," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 47(1), pages 91-113, February.
    41. Armstrong, J. Scott & Graefe, Andreas, 2011. "Predicting elections from biographical information about candidates: A test of the index method," Journal of Business Research, Elsevier, vol. 64(7), pages 699-706, July.
    42. D Johnson, 2002. "Triangular approximations for continuous random variables in risk analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(4), pages 457-467, April.
    43. Bruno Biais & Denis Hilton & Karine Mazurier & Sébastien Pouget, 2005. "Judgemental Overconfidence, Self-Monitoring, and Trading Performance in an Experimental Financial Market," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(2), pages 287-312.
    44. Nikolaeva, Ralitza, 2014. "Interorganizational imitation heuristics arising from cognitive frames," Journal of Business Research, Elsevier, vol. 67(8), pages 1758-1765.
    45. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    46. Christian Terwiesch & Yi Xu, 2008. "Innovation Contests, Open Innovation, and Multiagent Problem Solving," Management Science, INFORMS, vol. 54(9), pages 1529-1543, September.
    47. Yaniv, Ilan, 1997. "Weighting and Trimming: Heuristics for Aggregating Judgments under Uncertainty," Organizational Behavior and Human Decision Processes, Elsevier, vol. 69(3), pages 237-249, March.
    48. Fifić, Mario & Gigerenzer, Gerd, 2014. "Are two interviewers better than one?," Journal of Business Research, Elsevier, vol. 67(8), pages 1771-1779.
    49. Donald L. Keefer & Samuel E. Bodily, 1983. "Three-Point Approximations for Continuous Random Variables," Management Science, INFORMS, vol. 29(5), pages 595-609, May.
    50. Graefe, Andreas, 2015. "Improving forecasts using equally weighted predictors," Journal of Business Research, Elsevier, vol. 68(8), pages 1792-1799.
    51. Cuzán, Alfred G. & Bundrick, Charles M., 2009. "Predicting Presidential Elections with Equally Weighted Regressors in Fair's Equation and the Fiscal Model," Political Analysis, Cambridge University Press, vol. 17(3), pages 333-340, July.
    52. Mahmoud, Essam, 1989. "Combining forecasts: Some managerial issues," International Journal of Forecasting, Elsevier, vol. 5(4), pages 599-600.
    53. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    54. Hibon, Michele & Evgeniou, Theodoros, 2005. "To combine or not to combine: selecting among forecasts and their combinations," International Journal of Forecasting, Elsevier, vol. 21(1), pages 15-24.
    55. Spyros Makridakis, 1990. "Note---Sliding Simulation: A New Approach to Time Series Forecasting," Management Science, INFORMS, vol. 36(4), pages 505-512, April.
    56. Yates, J. Frank & McDaniel, Linda S. & Brown, Eric S., 1991. "Probabilistic forecasts of stock prices and earnings: The hazards of nascent expertise," Organizational Behavior and Human Decision Processes, Elsevier, vol. 49(1), pages 60-79, June.
    57. Sushil Bikhchandani & David Hirshleifer & Ivo Welch, 1998. "Learning from the Behavior of Others: Conformity, Fads, and Informational Cascades," Journal of Economic Perspectives, American Economic Association, vol. 12(3), pages 151-170, Summer.
    58. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    59. Jay F. Nunamaker & Amit V. Deokar, 2008. "GDSS Parameters and Benefits," International Handbooks on Information Systems, in: Handbook on Decision Support Systems 1, chapter 20, pages 391-414, Springer.
    60. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, June.
    61. Gerardine DeSanctis & R. Brent Gallupe, 1987. "A Foundation for the Study of Group Decision Support Systems," Management Science, INFORMS, vol. 33(5), pages 589-609, May.
    62. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    63. Mark J. Flannery & Aris A. Protopapadakis, 2002. "Macroeconomic Factors Do Influence Aggregate Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 15(3), pages 751-782.
    64. McKenzie, Craig R.M. & Liersch, Michael J. & Yaniv, Ilan, 2008. "Overconfidence in interval estimates: What does expertise buy you?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(2), pages 179-191, November.
    65. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    66. Onkal, Dilek & Yates, J. Frank & Simga-Mugan, Can & Oztin, Sule, 2003. "Professional vs. amateur judgment accuracy: The case of foreign exchange rates," Organizational Behavior and Human Decision Processes, Elsevier, vol. 91(2), pages 169-185, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    3. Bacci, Livio Agnew & Mello, Luiz Gustavo & Incerti, Taynara & Paulo de Paiva, Anderson & Balestrassi, Pedro Paulo, 2019. "Optimization of combined time series methods to forecast the demand for coffee in Brazil: A new approach using Normal Boundary Intersection coupled with mixture designs of experiments and rotated fact," International Journal of Production Economics, Elsevier, vol. 212(C), pages 186-211.
    4. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    5. Diebold, Francis X. & Shin, Minchul, 2019. "Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1679-1691.
    6. João F. Caldeira & Guilherme V. Moura & Francisco J. Nogales & André A. P. Santos, 2017. "Combining Multivariate Volatility Forecasts: An Economic-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 15(2), pages 247-285.
    7. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    8. Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015. "Optimal combination of survey forecasts," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
    9. Graefe, Andreas & Armstrong, J. Scott & Jones, Randall J. & Cuzán, Alfred G., 2014. "Combining forecasts: An application to elections," International Journal of Forecasting, Elsevier, vol. 30(1), pages 43-54.
    10. Li, Li & Kang, Yanfei & Li, Feng, 2023. "Bayesian forecast combination using time-varying features," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1287-1302.
    11. Kang, Yanfei & Spiliotis, Evangelos & Petropoulos, Fotios & Athiniotis, Nikolaos & Li, Feng & Assimakopoulos, Vassilios, 2021. "Déjà vu: A data-centric forecasting approach through time series cross-similarity," Journal of Business Research, Elsevier, vol. 132(C), pages 719-731.
    12. Kourentzes, Nikolaos & Barrow, Devon & Petropoulos, Fotios, 2019. "Another look at forecast selection and combination: Evidence from forecast pooling," International Journal of Production Economics, Elsevier, vol. 209(C), pages 226-235.
    13. Xiaojie Xu, 2020. "Corn Cash Price Forecasting," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(4), pages 1297-1320, August.
    14. Thomson, Mary E. & Pollock, Andrew C. & Önkal, Dilek & Gönül, M. Sinan, 2019. "Combining forecasts: Performance and coherence," International Journal of Forecasting, Elsevier, vol. 35(2), pages 474-484.
    15. Nowotarski, Jakub & Raviv, Eran & Trück, Stefan & Weron, Rafał, 2014. "An empirical comparison of alternative schemes for combining electricity spot price forecasts," Energy Economics, Elsevier, vol. 46(C), pages 395-412.
    16. Blanc, Sebastian M. & Setzer, Thomas, 2016. "When to choose the simple average in forecast combination," Journal of Business Research, Elsevier, vol. 69(10), pages 3951-3962.
    17. Graefe, Andreas, 2023. "Embrace the differences: Revisiting the PollyVote method of combining forecasts for U.S. presidential elections (2004 to 2020)," International Journal of Forecasting, Elsevier, vol. 39(1), pages 170-177.
    18. Lyon, Aidan & Wintle, Bonnie C. & Burgman, Mark, 2015. "Collective wisdom: Methods of confidence interval aggregation," Journal of Business Research, Elsevier, vol. 68(8), pages 1759-1767.
    19. Karsten R. Gerdrup & Anne Sofie Jore & Christie Smith & Leif Anders Thorsrud, 2009. "Evaluating ensemble density combination - forecasting GDP and inflation," Working Paper 2009/19, Norges Bank.
    20. Kourentzes, Nikolaos & Petropoulos, Fotios & Trapero, Juan R., 2014. "Improving forecasting by estimating time series structural components across multiple frequencies," International Journal of Forecasting, Elsevier, vol. 30(2), pages 291-302.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:69:y:2016:i:10:p:4081-4094. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.