Pitfalls in the Development of Falsification Tests: An Illustration from the Recent Minimum Wage Literature

This paper examines a ``falsification test'' from the recent minimum wage literature. The analysis illustrates several pitfalls associated with developing and interpreting such exercises, which are increasingly common in applied empirical work. Clemens and Wither (2014) present evidence that minimum wage increases contributed to the magnitude of employment declines among low-skilled groups during the Great Recession. Zipperer (2016) presents regressions that he interprets as falsification tests for Clemens and Wither's baseline regression. He interprets his results as evidence that Clemens and Wither's estimates are biased. In this paper, I demonstrate that Zipperer's falsification tests are uninformative for their intended purpose. The properties of clustered robust standard errors do not carry over from Clemens and Wither's baseline specification (27 treatment states drawn from 50) to Zipperer's falsification tests (3 or 5 ``placebo treatment'' states drawn from 23). Confidence intervals calculated using a setting-appropriate permutation test extend well beyond the tests' point estimates. Further, I show that the sub-samples to which Zipperer's procedure assigns ``placebo treatment status'' were disproportionately affected by severe housing crises. His test's point estimates are highly sensitive to the exclusion of the most extreme housing crisis experiences from the sample. An inspection of data on the housing market, prime aged employment, overall unemployment rates, and aggregate income per capita reveals the test's premise that regional neighbors form reasonable counterfactuals to be incorrect in this setting.