ANVILS-VOCE: ANova-based Varying Inner-Loop Size estimation of Variance of Conditional Expectation

Analysis of Variance (ANOVA) is a popular method to infer, based on the sampled data, whether the true means of a set of subpopulations differ from each other. The variance of conditional expectation (VOCE) is the variance of these effects in sub-populations, and this is estimated by sampling a sub-population of size nk, for each sub-population labeled k, and by sampling K such sub-populations in the experiment. For the general case of varying nk, it is unknown what the variance of the VOCE estimator is, though it is known for the special case nk=n, n≥2 for all k∈1,2,…,K as derived in the literature. The following derivation settles the former question and is of value in situations where sampling has constraints or only a skewed sampling budget is available. Our first application is with regard to the decision of whether samples from pilot simulation can be included in the regular simulation to estimate VOCE. The second application is an estimation technique where the estimate of optimal inner-loop size n* can be updated throughout the duration of simulation. We demonstrate with these 2 applications where we observe a 20% reduction in the variance of the VOCE estimate, when the proposed method is applied.