Parallel Adaptive Survivor Selection

We reconsider the ranking and selection (R&S) problem in stochastic simulation optimization in light of high-performance, parallel computing, where we take “R&S” to mean any procedure that simulates all systems (feasible solutions) to provide some statistical guarantee on the selected systems. We argue that when the number of systems is very large, and the parallel processing capability is also substantial, then neither the standard statistical guarantees such as probability of correct selection nor the usual observation-saving methods such as elimination via paired comparisons or complex budget allocation serve the experimenter well. As an alternative, we propose a guarantee on the expected false elimination rate that avoids the curse of multiplicity and a method to achieve it that is designed to scale computationally with problem size and parallel computing capacity . To facilitate this approach, we present a new mathematical representation, prove small-sample and asymptotic properties, evaluate variations of the method, and demonstrate a specific implementation on a problem with over 1 , 100 , 000 systems using only 21 parallel processors. Although we focus on inference about the best system here, our parallel adaptive survivor selection framework can be generalized to many other useful definitions of “good” systems.