Evaluating the Quality of Research into a Single Prognostic Biomarker: A Systematic Review and Meta-analysis of 83 Studies of C-Reactive Protein in Stable Coronary Artery Disease

In a systematic review and meta-analysis of 83 prognostic studies of C-reactive protein in coronary disease, Hemingway and colleagues find substantial biases, preventing them from drawing clear conclusions relating to the use of this marker in clinical practice.Background: Systematic evaluations of the quality of research on a single prognostic biomarker are rare. We sought to evaluate the quality of prognostic research evidence for the association of C-reactive protein (CRP) with fatal and nonfatal events among patients with stable coronary disease. Methods and Findings: We searched MEDLINE (1966 to 2009) and EMBASE (1980 to 2009) and selected prospective studies of patients with stable coronary disease, reporting a relative risk for the association of CRP with death and nonfatal cardiovascular events. We included 83 studies, reporting 61,684 patients and 6,485 outcome events. No study reported a prespecified statistical analysis protocol; only two studies reported the time elapsed (in months or years) between initial presentation of symptomatic coronary disease and inclusion in the study. Studies reported a median of seven items (of 17) from the REMARK reporting guidelines, with no evidence of change over time. Conclusion: Multiple types of reporting bias, and publication bias, make the magnitude of any independent association between CRP and prognosis among patients with stable coronary disease sufficiently uncertain that no clinical practice recommendations can be made. Publication of prespecified statistical analytic protocols and prospective registration of studies, among other measures, might help improve the quality of prognostic biomarker research. : Please see later in the article for the Editors' Summary Background: Coronary artery disease is the leading cause of death among adults in developed countries. With age, fatty deposits called atherosclerotic plaques coat the walls of the arteries, the vessels that carry blood to the body's organs. Because they narrow the arteries, atherosclerotic plaques restrict blood flow. If plaques form in the arteries that feed the heart, the result is coronary artery disease, the symptoms of which include shortness of breath and chest pains (angina). If these symptoms only occur during exertion, the condition is called stable coronary artery disease. Coronary artery disease can cause potentially fatal heart attacks (myocardial infarctions). A heart attack occurs when a plaque ruptures and a blood clot completely blocks the artery, thereby killing part of the heart. Smoking, high blood pressure, high blood levels of cholesterol (a type of fat), diabetes, and being overweight are risk factors for coronary artery disease. Treatments for the condition include lifestyle changes and medications that lower blood pressure and blood cholesterol. Narrowed arteries can also be widened using a device called a stent or surgically bypassed. Why Was This Study Done?: Clinicians can predict whether a patient with coronary artery disease is likely to have a heart attack by considering their risk factors. They then use this “prognosis” to help them manage the patient. To provide further help for clinicians, researchers are trying to identify prognostic biomarkers (molecules whose blood levels indicate how a disease might develop) for coronary artery disease. However, before a biomarker can be used clinically, it must be properly validated and there are concerns that there is insufficient high quality evidence to validate many biomarkers. In this systematic review and meta-analysis, the researchers ask whether the evidence for an association between blood levels of C-reactive protein (CRP, an inflammatory protein) and subsequent fatal and nonfatal events affecting the heart and circulation (cardiovascular events) among patients with stable coronary artery disease supports the routine measurement of CRP as recommended in clinical practice guidelines. A systematic review uses predefined criteria to identify all the research on a given topic; a meta-analysis is a statistical method for combining the results of several studies. What Did the Researchers Do and Find?: The researchers identified 83 studies that investigated the association between CRP levels measured in people with coronary artery disease and subsequent cardiovascular events. Their examination of these studies revealed numerous reporting and publication short-comings. For example, none of the studies reported a prespecified statistical analysis protocol, yet analyses should be prespecified to avoid the choice of analytical method biasing the study's results. Furthermore, on average, the studies only reported seven of the 17 recommended items in the REMARK reporting guidelines, which were designed to improve the reporting quality of tumor biomarker prognostic studies. The meta-analysis revealed that patients with a CRP level in the top third of the distribution were nearly twice as likely to have a cardiovascular event as patients with a CRP in the bottom third of the distribution (a relative risk of 1.97). However, the outcomes varied considerably between studies (heterogeneity) and there was strong evidence for publication bias—most published studies were small and smaller studies were more likely to report higher relative risks. Adjustment for publication bias reduced the relative risk associated with high CRP levels to 1.19. Finally, nearly all the studies failed to calculate whether CRP measurements discriminated between patients likely and unlikely to have a subsequent cardiovascular event. What Do These Findings Mean?: These findings suggest that, because of multiple types of reporting and publication bias, the size of the association between CRP levels and prognosis among patients with stable coronary artery disease is extremely uncertain. They also suggest that CRP measurements are unlikely to add anything to the prognostic discrimination achieved by considering blood pressure and other standard clinical factors among this patient group. Thus, the researchers suggest, the recommendation that CRP measurements should be used in the management of patients with stable coronary artery disease ought to be removed from clinical practice guidelines. More generally, these findings increase concerns about the quality of research into prognostic biomarkers and highlight areas that need to be changed, the most fundamental of which is the need to preregister studies on prognostic biomarkers and their analytic protocols. Additional Information: Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000286.