Multivariate tests comparing binomial probabilities, with application to safety studies for drugs

Summary. In magazine advertisements for new drugs, it is common to see summary tables that compare the relative frequency of several side‐effects for the drug and for a placebo, based on results from placebo‐controlled clinical trials. The paper summarizes ways to conduct a global test of equality of the population proportions for the drug and the vector of population proportions for the placebo. For multivariate normal responses, the Hotelling T2‐test is a well‐known method for testing equality of a vector of means for two independent samples. The tests in the paper are analogues of this test for vectors of binary responses. The likelihood ratio tests can be computationally intensive or have poor asymptotic performance. Simple quadratic forms comparing the two vectors provide alternative tests. Much better performance results from using a score‐type version with a null‐estimated covariance matrix than from the sample covariance matrix that applies with an ordinary Wald test. For either type of statistic, asymptotic inference is often inadequate, so we also present alternative, exact permutation tests. Follow‐up inferences are also discussed, and our methods are applied to safety data from a phase II clinical trial.