The Role of Theory in Experimental Design: Experiments Without Randomization

Some who have written about the logic of experimentation argue that random assignment of subjects to treatment conditions is an essential attribute of an experiment. Others disagree. Rather than treating this as a matter of dueling definitions, we consider experiments without randomization from a theoretical perspective. Our central contention here is that, for some research questions, theory dictates systematic (not random) assignment of respondents to experimental conditions. Two such areas of inquiry are research on political tolerance and on institutional legitimacy. This article gives cursory attention to the former body of work and detailed attention to the latter, based on an experiment conducted in a survey in 2001 on the consequences of the American presidential election for institutional legitimacy. Because in both instances theory requires nonrandom assignment, the problem becomes one of identifying the costs of nonrandomization (threats to internal validity) and specifying analytical techniques that might ameliorate those costs. Consequently, we present results from a statistical approach that addresses the problem of nonrandomization. The most important claim of this article is that theory ought to specify research design, including experimental designs, and that dogmatic attachment to one definition of experiment will not serve the discipline of political science.