What is the external validity of sentencing research? A multi-level meta-analysis of race and gender disparities

Sentencing research is rarely cross-jurisdictional. More problematically, most quantitative sentencing research is based on a limited number of American jurisdictions where court data is available. As a result, it is difficult to assess the extent to which key findings from the sentencing literature apply universally. We build on the recent growth of sentencing research outside the US to explore the external validity of studies reporting the conditional association of offenders’ race and gender with sentence length. To do so, we conduct two multi-level meta-analyses, distinguishing the proportion of between-study heterogeneity attributable to differences at the study and jurisdiction levels. Our findings reveal that while race disparities in sentencing are statistically significant, they are minimal in magnitude (a 3% penalty for racial minorities) and remarkably consistent across jurisdictions. In contrast, gender disparities are more pronounced (a 13% penalty against men) but highly variable, with some jurisdictions showing parity. Both analyses uncover substantial variability due to sample and modelling choices, highlighting the limited generalisability of existing sentencing research. We urge caution in interpreting findings from single studies on sentencing disparities and advocate for the pre-registration of analytical strategies to mitigate researcher bias.