Choosing the Devil You Don’t Know: Evidence for Limited Sensitivity to Sample Size–Based Uncertainty When It Offers an Advantage

Many decision makers seek to optimize choices between uncertain options such as strategies, employees, or products. When performance targets must be met, attending to observed past performance is not enough to optimize choices—option uncertainty must also be considered. For example, for stretch targets that exceed observed performance, more uncertain options are often better bets. A significant determinant of option uncertainty is sample size: for a given option, the smaller the sample of information we have about it, the greater the uncertainty. In two studies, choices were made between pairs of uncertain options with the goal of exceeding a specified performance target. Information about the options differed in the size of the sample drawn from them, sample size , and the observed performance of those samples, the proportion of successes or “hits” in the sample. We found people to be sensitive to sample size–based uncertainty only when differences in observed performance were negligible. We conclude that in the presence of performance targets, people largely fail to capitalize on the value advantages of small samples in the presence of stretch targets.