Optimal exact tests for multiple binary endpoints

In confirmatory clinical trials with small sample sizes, hypothesis tests based on asymptotic distributions are often not valid and exact non-parametric procedures are applied instead. However, the latter are based on discrete test statistics and can become very conservative, even more so, if adjustments for multiple testing as the Bonferroni correction are applied. Improved exact multiple testing procedures are proposed for the setting where two parallel groups are compared in multiple binary endpoints. Based on the joint conditional distribution of test statistics of Fisher’s exact tests, optimal rejection regions for intersection hypothesis tests are constructed utilizing different objective functions. Depending on the optimization objective, the optimal test yields maximal power under a specific alternative, maximal exhaustion of the nominal type I error rate, or the largest possible rejection region controlling the type I error rate. To efficiently search the large space of possible rejection regions, an optimization algorithm based on constrained optimization and integer linear programming is proposed. Applying the closed testing principle, optimized multiple testing procedures with strong familywise error rate control are constructed. Furthermore, a computationally efficient greedy algorithm for nearly optimal tests is proposed. The unconditional power of the optimized procedures is numerically compared to the power of alternative approaches and the optimal tests are illustrated with a clinical trial example in a rare disease. The described methods are implemented in the R package multfisher.