Can we estimate crisis death tolls by subtracting population estimates? A critical review and appraisal

In the absence of high-quality data, death tolls are often estimated using ad hoc methods. One understudied class of methods, which we term the growth rate discontinuity method (GRDM), estimates the size of death tolls by projecting between pre- and post-crisis population estimates with crude growth rates and then subtracting the projected values. Despite its simplicity, this method is the source of prominent death toll estimates for the Black Death, the 1918 influenza pandemic, the Great Chinese Famine, and the Rwandan genocide, among others. In this article, we review the use and validity of GRDM. Using statistical simulation and comparisons to the results of well-validated demographic methods, we assess the accuracy, precision, and biases of this method for estimating mortality in absolute and relative terms. We find that GRDM requires precision in its inputs to an extent rarely possible in contexts of interest. If one has sufficient data to specify GRDM well, one can likely also use a more reliable method; if one lacks that data, the estimates produced by the method are so sensitive to data quality issues and other assumptions that they cannot be considered informative. These findings create problems for existing demographic and econometric work that has used GRDM to analyze mortality crises for which demographic data is scarce.