Evaluating Public Commentary and Scientific Evidence Submitted in the Development of a Risk Assessment

Risk assessments form the methodological basis for many public policies. A key component of the risk assessment process is the public commentary period. We conducted a case study of the California environmental tobacco smoke risk assessment to describe the contribution of the commentary to the risk assessment process. We used content analysis to examine the sources, quantity, and quality of public commentary, as well as the agency's response to the commentary. We examined the type and quality of publications cited in the commentary. Most of the comments were from critics of the risk assessment (36/44, 80%), especially tobacco industry affiliates (30/36, 83%). Critics were more likely to evoke the science evaluation criteria of study quality, reliability, and validity than were supporters. More than half the critics argued that appropriate procedures were not followed (13/23, 57%). Of the 29 commentaries on the respiratory, carcinogenic, and cardiovascular chapters, four resulted in changes to the risk assessment, such as the addition of new references or reanalysis of data. Journal articles were the most frequently cited type of reference, cited by critics (1,022/1,526 of references, 67%) and supporters (39/60, 65%). However, journal articles submitted by critics had lower impact factors than those cited by supporters (2.6 vs. 3.6, p=0.03). Participation in the public input process was not balanced among all interested parties, although this may reflect different opportunities for stakeholders to participate in stages of the process. Critics and supporters of the risk assessment used different criteria to evaluate the scientific evidence, suggesting that they were socially constructing the evidence to support their positions.