Artifactual evidence of discrimination in correspondence studies? A replication of the Neumark method

The advocates of correspondence testing (CT) argue that it provides the most clear and convincing evidence of discrimination. The common view is that the standard CT can identify what is typically defined as discrimination in a legal sense - what we label total discrimination in the current study -, although it cannot separate between preferences and statistical discrimination. However, Heckman and Siegelman (1993) convincingly show that audit and correspondence studies can obtain biased estimates of total discrimination - in any direction - if employers evaluate applications according to some threshold level of productivity. This issue has essentially been ignored in the empirical literature on CT experiments until the appearance of the methodology proposed by Neumark (2012). He shows that with the right data and an identifying assumption, with testable predictions, this method can identify the total discrimination. In the current paper we use this new method to reexamine a number of already published correspondence studies to investigate whether their estimate of total discrimination is affected by group differences in variances of unobservable characteristics. We also aim at improving the general understanding of to what extent the standardization level of job applications is an issue in empirical work. We find that the standardization level of the job applications being set by the experimenter appear to be a general issue in correspondence studies which must be taken seriously.