ANOVA with two timescale stochastic approximation for estimating Variance of Conditional Expectation

The ANOVA method is of value to detect if a population, consisting of labelled sub-populations, hasany statistically significant support for considering such labels as valid. In classical ANOVA, the effectof a variable in each sub-population is treated as a Conditional Expectation (CE), and the variance ofsuch CE among the sub-populations has a bearing on whether the null hypothesis can be rejected or not.ANOVA formulae can therefore be used to estimate the Variance of CE (Var-of-CE) itself, and a fairly recentpublication has proposed a method wherein a fixed number of samples in each sub-population is used toestimate Var-of-CE. This method assumes repeated sampling of both sub-populations and samples withinthem, and have designed provably unbiased estimators of Var-of-CE, with one of these being approximatelyminimum variance under some conditions. Combined with another more recent method, such methods havedisadvantages, such as requiring a pilot simulation, or suffering an empirically-observed Root Mean SquaredError (RMSE) that is unfavourable. The work explained here proposes an ANOVA estimator for Var-of-CEthat requires an increasing number of samples from each subpopulation. Yet, the estimator reduces theempirically-observed MSE in Var-of-CE estimate in3benchmark experiments from the literature.