Comparing distributions by multiple testing across quantiles or CDF values

We first show that one-sample and two-sample Kolmogorov–Smirnov tests may be interpreted as multiple testing procedures, nonparametrically testing equality at each point in the distribution with strong control of the finite-sample familywise error rate. Second, we provide an alternative procedure that distributes power across the distribution more evenly than the Kolmogorov–Smirnov test, which suffers low sensitivity to tail deviations. Third, we provide a formula for near-instant one-sample computation. Fourth, we improve power with stepdown and pre-test procedures. Finally, we extend our results to conditional distributions and regression discontinuity designs. Simulations, empirical examples, and code are provided.