Signaling Class: An Examination of the Treatment Validity of Names Used to Signal Race in Bias Experiments with Methodological Recommendations for Name Selection

Racial bias experiments commonly use names to signal race as treatments. However, recent methodological examinations find that individuals often perceive class and race together. This calls into question the treatment validity of thousands of experiments. Still, little evidence exists on what leads to name perceptions and how scholars might increase treatment validity in future studies. I suggest that racialized and classed demographic naming patterns may influence individuals’ perceptions of names. I conducted two survey experiments and used demographic birth record data to examine social class perceptions. In total, 7,695 respondents provided 82,321 perceptions on 636 combinations of first and last names. Although demographic naming patterns have small effects on respondents’ social class perceptions of White-signaled names, classed patterns have a large effect on respondents’ perceptions of Black-signaled names. These findings suggest that treatment validity is a severe problem for bias experiments. To help mitigate this problem, I provide seven recommendations that researchers should implement in all experiments that use names to signal various characteristics. Scholars who follow these recommendations will neutralize or minimize threats to treatment validity, engage in a more empirical and open scientific process, and, in some cases, open up new avenues of research on bias.