Measuring Coverage in MNCH: A Prospective Validation Study in Pakistan and Bangladesh on Measuring Correct Treatment of Childhood Pneumonia

Background: Antibiotic treatment for pneumonia as measured by Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) is a key indicator for tracking progress in achieving Millennium Development Goal 4. Concerns about the validity of this indicator led us to perform an evaluation in urban and rural settings in Pakistan and Bangladesh. Methods and Findings: Caregivers of 950 children under 5 y with pneumonia and 980 with “no pneumonia” were identified in urban and rural settings and allocated for DHS/MICS questions 2 or 4 wk later. Study physicians assigned a diagnosis of pneumonia as reference standard; the predictive ability of DHS/MICS questions and additional measurement tools to identify pneumonia versus non-pneumonia cases was evaluated. Conclusions: Monitoring antibiotic treatment of pneumonia is essential for national and global programs. Current (DHS/MICS questions) and proposed new (video and pneumonia score) methods of identifying pneumonia based on maternal recall discriminate poorly between pneumonia and children with cough. Furthermore, these methods have a low yield to identify children who have true pneumonia. Reported antibiotic treatment rates among these children are therefore not a valid proxy indicator of pneumonia treatment rates. These results have important implications for program monitoring and suggest that data in its current format from DHS/MICS surveys should not be used for the purpose of monitoring antibiotic treatment rates in children with pneumonia at the present time. Background: Pneumonia is a major cause of death in children younger than five years across the globe, with approximately 1.2 million children younger than five years dying from pneumonia every year. Pneumonia can be caused by bacteria, fungi, or viruses. It is possible to effectively treat bacterial pneumonia with appropriate antibiotics; however, only about 30% of children receive the antibiotic treatment they need. The Millennium Development Goals (MDGs) are eight international development goals that were established in 2000. The fourth goal (MDG 4) aims to reduce child mortality, specifically, to reduce the under-five mortality rate by two-thirds, between 1990 and 2015. Given that approximately 18% of all deaths in children under five are caused by pneumonia, providing universal coverage with effective treatments for pneumonia is an important part of MDG 4. Why Was This Study Done?: Concerns have been raised about whether information collected from DHS and MICS is able to accurately identify cases of pneumonia. In a clinical setting, pneumonia is typically diagnosed based on a combination of physical symptoms, including coughing, rapid breathing, or difficulty breathing, and a chest X-ray. The surveys rely on information collected from interviews of mothers and primary caregivers using structured questions about whether the child has experienced physical symptoms in the past two weeks and whether these were chest-related. The DHS survey labels this condition as “symptoms of acute respiratory infection,” while the MICS survey uses the term “suspected pneumonia.” Thus, these surveys provide a proxy measure for pneumonia that is limited by the reliance on the recall of symptoms by the mother or caregiver. Here the researchers have evaluated the use of these surveys to discriminate physician-diagnosed pneumonia and to provide accurate recall of antibiotic treatment in urban and rural settings in Pakistan and Bangladesh. What Did the Researchers Do and Find?: The researchers identified caregivers of 950 children under five years with pneumonia and 980 who had a cough or cold but did not have pneumonia from urban and rural settings in Pakistan and Bangladesh. Cases of pneumonia were identified based on a physician diagnosis using World Health Organization guidelines. They randomly assigned caregivers to be interviewed using DHS and MICS questions with either a two- or four-week recall period. They then assessed how well the DHS and MICS questions were able to accurately diagnose pneumonia and accurately recall antibiotic use. In addition, they asked caregivers to complete a pneumonia score questionnaire and showed them a video tool showing children with and without pneumonia, as well as a medication drug chart, to determine if these alternative measures improved the accuracy of pneumonia diagnosis or recall of antibiotic use. They found that both surveys, the pneumonia score, and the video tool had poor ability to discriminate between children with and without physician-diagnosed pneumonia, and there were no differences between using two- or four-week recall. The sensitivity (proportion of pneumonia cases that were correctly identified) ranged from 23% to 72%, and the specificity (the proportion of “no pneumonia” cases that were correctly identified) ranged from 53% to 83%, depending on the setting. They also observed that prescribed antibiotics for pneumonia were correctly recalled by about two-thirds of caregivers using DHS questions, and this increased to about three-quarters of caregivers when using a drug chart and detailed enquiry. What Do These Findings Mean?: The findings of this study suggest that the current use of questions from DHS and MICS based on mother or caregiver recall are not sufficient for accurately identifying pneumonia and antibiotic use in children. Because these surveys have poor ability to identify children who have true pneumonia, reported antibiotic treatment rates for children with pneumonia based on data from these surveys may not be accurate, and these surveys should not be used to monitor treatment rates. These findings should be interpreted cautiously, given the relatively high rate of loss to follow-up and delayed follow-up in some of the children and because some of the settings in this study may not be similar to other low-income settings. Additional Information: Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001422.