Is Satisfaction Success? Evaluating Public Participation in Regulatory Policymaking

Researchers who evaluate ordinary dispute resolution procedures understandably ask whether disputants are eventually satisfied with the resulting outcomes. But how should they evaluate techniques for involving the public in regulatory policymaking, i.e., techniques such as comment solicitation, public hearings, workshops, dialogue groups, advisory committees, and negotiated rulemaking processes? Researchers evaluating these various techniques have also typically used participant satisfaction as a key evaluative criterion. While this criterion may seem suitable for evaluating private dispute resolution techniques, those who disagree in policy-making processes are not disputants in the same sense that landlords and tenants, creditors and debtors, or tortfeasors and victims are disputants in private life. Disputes in regulatory policymaking arise over public policy, not over private grievances. This paper raises caution about using participant satisfaction, or other measures based on participants’ attitudes and opinions, in evaluating dispute resolution and public participation in regulatory policymaking. It argues that a reliance on participant satisfaction encounters two conceptual limitations, namely that (a) participant satisfaction does not necessarily equate with good public policy and (b) participant satisfaction is at best an incomplete measure because it excludes those who do not participate. In addition, the paper outlines a series of problems in applying, measuring, and interpreting participant satisfaction that make it a problematic metric for evaluating public participation in regulatory processes. In light of the conceptual and measurement problems with relying on satisfaction, evaluation researchers should resist relying on participant surveys to evaluate public participation techniques, and instead should focus attention directly on the effectiveness, efficiency, and equity of the decisions that result from different forms of public participation.