Reverse Regressions, Symmetry and Test Distributions in Linear Models

This paper extends the applicability of normal distributional theory in linear regression. In the classical linear model, the t statistic follows the well-known Student t distribution. Inspired by C. R. Rao’s characterization results, we give new conditions under which this result holds even if the errors are not i.i.d. normal. This is based on observing a novel symmetry: on considering two linear regressions, the direct regression (regression of y on x and Z) and the reverse regression (regression of x on y and Z), the t statistic ( $$t_{D}$$ t D ) for the coefficient of x in the direct regression is numerically identical to the t statistic for y in the reverse regression (regression of x on y and Z). This implies that their distributions are also identical. We show this yields a new type of condition under which a t statistic follows the usual Student t distribution, without an assumption on the conditional distribution of the dependent variable given the regressors. We extend these results to F statistics as well as various statistics in multivariate linear regressions. A simulation study confirms our theoretical results. We also present examples including field experiments, simultaneous equations models, and analysis of discrimination, to which our findings can be applied.