A Retrospective Review of Retrospective Cost Analyses

In this paper, we identify the main factors that drive differences between ex ante and ex post cost estimates. The paper reviews evidence from peer-reviewed studies of the realized costs of 13 significant EPA regulations to develop lessons for the design of future ex ante and ex post analyses of rules. Most of the retrospective studies address realized compliance strategies, though some also offer insights into specific elements of per-unit compliance costs. Only a few shed light on the total cost of the regulation studied. All the studies have data limitations, but in general, more insight was obtained when detailed facility-level data (e.g., power plants) were available. In spite of data and methodological limitations, as well as the narrow focus of some studies, we identify several common sources of differences between ex ante and ex post estimates. For example, these studies reveal that firms adopted substantially different compliance strategies than anticipated in ex ante analysis for nearly 70 percent of the regulations. Other key drivers of differences include reliance on engineering models, misspecification of the baseline, and failure to anticipate the role of new technologies. To improve future ex ante cost analysis, we recommend better characterization of baseline conditions, sensitivity analysis of highly uncertain parameters, greater use of economic models of the regulated sector to better reflect firm decision making, and analysis of phase-in periods. For ex post analyses, we recommend developing plans for future study at the time the regulation is adopted. Aside from opportunistic cases, it will be difficult to conduct thorough retrospective evaluations without a plan in place ex ante that identifies endpoints of interest, methods of analysis, and data needs.