Using Qualitative Information to Improve Causal Inference

Using the Rosenbaum (2002, 2009) approach to observational studies, we show how qualitative information can be incorporated into quantitative analyses to improve causal inference in three ways. First, by including qualitative information on outcomes within matched sets, we can ameliorate the consequences of the difficulty of measuring those outcomes, sometimes reducing p‐values. Second, additional information across matched sets enables the construction of qualitative confidence intervals on effect size. Third, qualitative information on unmeasured confounders within matched sets reduces the conservativeness of Rosenbaum‐style sensitivity analysis. This approach accommodates small to medium sample sizes in a nonparametric framework, and therefore it may be particularly useful for analyses of the effects of policies or institutions in a small number of units. We illustrate these methods by examining the effect of using plurality rules in transitional presidential elections on opposition harassment in 1990s sub‐Saharan Africa.