Getting the Right Mix of Experts

The Bayesian approach to combining expert opinions is well developed, providing a decision maker's posterior beliefs after receiving advice from people with deep knowledge in a given subject. A necessary part of these models is the inclusion of dependencies between the experts' judgments, often justified by an overlap in the information on which the experts base their judgments. In this paper, we propose a hierarchical structure different than those previously proposed, where the mixing distribution is treated nonparametrically with a Dirichlet process. This makes our overall model a Dirichlet process mixture and allows for experts' model parameters to be equal in the mixture. We apply this approach to published expert judgment data, demonstrating that the decision maker's posterior distributions on the quantities of interest are not restricted to specific parametric forms, even allowing for multiple modes, and are thus more intuitively appealing.