Validation of the Symptom Pattern Method for Analyzing Verbal Autopsy Data

Background: Cause of death data are a critical input to formulating good public health policy. In the absence of reliable vital registration data, information collected after death from household members, called verbal autopsy (VA), is commonly used to study causes of death. VA data are usually analyzed by physician-coded verbal autopsy (PCVA). PCVA is expensive and its comparability across regions is questionable. Nearly all validation studies of PCVA have allowed physicians access to information collected from the household members' recall of medical records or contact with health services, thus exaggerating accuracy of PCVA in communities where few deaths had any interaction with the health system. In this study we develop and validate a statistical strategy for analyzing VA data that overcomes the limitations of PCVA. Methods and Findings: We propose and validate a method that combines the advantages of methods proposed by King and Lu, and Byass, which we term the symptom pattern (SP) method. The SP method uses two sources of VA data. First, it requires a dataset for which we know the true cause of death, but which need not be representative of the population of interest; this dataset might come from deaths that occur in a hospital. The SP method can then be applied to a second VA sample that is representative of the population of interest. From the hospital data we compute the properties of each symptom; that is, the probability of responding yes to each symptom, given the true cause of death. These symptom properties allow us first to estimate the population-level cause-specific mortality fractions (CSMFs), and to then use the CSMFs as an input in assigning a cause of death to each individual VA response. Finally, we use our individual cause-of-death assignments to refine our population-level CSMF estimates. The results from applying our method to data collected in China are promising. At the population level, SP estimates the CSMFs with 16% average relative error and 0.7% average absolute error, while PCVA results in 27% average relative error and 1.1% average absolute error. At the individual level, SP assigns the correct cause of death in 83% of the cases, while PCVA does so for 69% of the cases. We also compare the results of SP and PCVA when both methods have restricted access to the information from the medical record recall section of the VA instrument. At the population level, without medical record recall, the SP method estimates the CSMFs with 14% average relative error and 0.6% average absolute error, while PCVA results in 70% average relative error and 3.2% average absolute error. For individual estimates without medical record recall, SP assigns the correct cause of death in 78% of cases, while PCVA does so for 38% of cases. Conclusions: Our results from the data collected in China suggest that the SP method outperforms PCVA, both at the population and especially at the individual level. Further study is needed on additional VA datasets in order to continue validation of the method, and to understand how the symptom properties vary as a function of culture, language, and other factors. Our results also suggest that PCVA relies heavily on household recall of medical records and related information, limiting its applicability in low-resource settings. SP does not require that additional information to adequately estimate causes of death. Chris Murray and colleagues propose and, using data from China, validate a new strategy for analyzing verbal autopsy data that combines the advantages of previous methods. Background.: All countries need to know the leading causes of death among their people. Only with accurate cause-of-death data can their public-health officials and medical professionals develop relevant health policies and programs and monitor how they affect the nation's health. In developed countries, vital registration systems record specific causes of death that have been certified by doctors for most deaths. But, in developing countries, vital registration systems are rarely anywhere near complete, a situation that is unlikely to change in the near future. An approach that is being used increasingly to get information on the patterns of death in poor countries is “verbal autopsy” (VA). Trained personnel interview household members about the symptoms the deceased had before his/her death, and the circumstances surrounding the death, using a standard form. These forms are then reviewed by a doctor, who assigns a cause of death from a list of codes called the International Classification of Diseases. This process is called physician-coded verbal autopsy (PCVA). Why Was This Study Done?: PCVA is a costly, time-consuming way of analyzing VA data and may not be comparable across regions, because it relies on the views of local doctors about the likely causes of death. In addition, although several studies have suggested that PCVA is reasonably accurate, such studies have usually included information collected from household members about medical records or contacts with health services. In regions where there is little contact with health services, PCVA may be much more inaccurate. Ideally what is needed is a method for assigning causes of death from VA data that does not involve physician review. In this study, the researchers have developed a statistical method—the symptom pattern (SP) method—for analyzing VA data and asked whether it can overcome the limitations of PCVA. What Did the Researchers Do and Find?: The SP method uses VA data collected about a group of patients for whom the true cause of death is known to calculate the probability for each cause of death that a household member will answer yes when asked about various symptoms. These so-called “symptom properties” can be used to calculate population cause-specific mortality fractions (CSMFs—the proportion of the population that dies from each disease) from VA data and, using a type of statistical analysis called Bayesian statistics, can be used to assign causes of deaths to individuals. When used with data from a VA study done in China, the SP method estimated population CSMFs with an average relative error of 16% (this measure indicates how much the estimated and true CSMFs deviate), whereas PCVA estimated them with an average relative error of 27%. At the individual level, the SP method assigned the correct cause of death in 83% of cases; PCVA was right only 69% of the time. Removing the medical record recall section of the VA data had little effect on the accuracy with which the two methods estimated population CSMFs. However, whereas the SP method still assigned the correct cause of death in 78% of individual cases, the PVCA did so in only 38% of cases. What Do These Findings Mean?: These findings suggest that the SP method for analyzing VA data can outperform PCVA at both the population and the individual level. In particular, the SP method may be much better than PCVA at assigning the cause of death for individuals who have had little contact with health services before dying, a common situation in the poorest regions of world. The SP method needs to be validated using data from other parts of the world and also needs to be tested in multi-country validation studies to build up information about how culture and language affect the likelihood of specific symptoms being reported in VAs for each cause of death. Provided the SP method works as well in other countries as it apparently does in China, its adoption, together with improvements in how VA data are collected, has the potential to improve the accuracy of cause-of-death data in developing countries. Additional Information.: Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0040327.