Metawisdom of the Crowd: How Choice Within Aided Decision Making Can Make Crowd Wisdom Robust

Quality information can improve individual judgments but nonetheless fail to make group decisions more accurate; if individuals choose to attend to the same information in the same way, the predictive diversity that enables crowd wisdom may be lost. Decision support systems, from business intelligence software to public search engines, present individuals with decision aids -- discrete presentations of relevant information, interpretative frames, or heuristics -- to enhance the quality and speed of decision making, but have the potential to bias judgments through the selective presentation of information and interpretative frames. We redescribe the wisdom of the crowd as often having two decisions, the choice of decision aids and then the primary decision. We then define \emph{metawisdom of the crowd} as any pattern by which the collective choice of aids leads to higher crowd accuracy than randomized assignment to the same aids, a comparison that accounts for the information content of the aids. While choice is ultimately constrained by the setting, in two experiments -- the prediction of inflation (N=947, pre-registered) and a tightly controlled estimation game (N=1198) -- we find strong evidence of metawisdom. It comes about through diverse errors arising through the use of diverse aids, not through widespread use of the aids that induce the most accurate estimates. Thus the microfoundations of crowd wisdom appear in the first choice, suggesting crowd wisdom can be robust in information choice problems. Given the implications for collective decision making, more research on the nature and use of decision aids is needed.