Experimental Evaluation of Variance Reduction Techniques for Queueing Simulation Using Generalized Concomitant Variables

In a companion paper we developed a unified scheme for using poststratified sampling and control variables to improve the efficiency of regenerative queueing simulations. We adapted these variance reduction techniques to the estimation methods of replication analysis and regenerative analysis by exploiting the asymptotic joint normality of certain standardized concomitant variables that are defined on each input process sampled during a queueing simulation. In this paper we present an experimental evaluation of the reductions in point-estimator variance and confidence-interval width that can be achieved with each procedure in several closed and mixed machine-repair systems. For the analytically tractable model, nominal and actual confidence-interval coverage probabilities are also compared. Poststratification generally produced variance reductions ranging from 10% to 40% and confidence-interval reductions between 1% and 20%. The control-variates schemes yielded variance reductions ranging from 20% to 90% and confidence-interval reductions between 10% and 70%. In some instances where small sample sizes were used, the poststratification schemes produced confidence-interval width increases between 1% and 10%. With a small number of regenerative cycles, some loss of confidence-interval coverage was observed with both post-stratified and controlled regenerative analysis. When larger sample sizes were used, all of the estimation schemes yielded fairly consistent efficiency gains.