Revising a Priority List Based on Cost-Effectiveness

Background. People sometimes object to the results of cost-effectiveness analysis when the analysis produces a ranking of options based on both cost and benefit. We suggest 2 new reasons for these objections: the prominence effect, in which people attend mostly to a more prominent attribute (benefit as opposed to cost), and distortion of utility judgments. Method. We simulated the production of a cost-effectiveness ranking list in 3 experiments using questionnaires on the World Wide Web. Subjects rated the utility of 16 health benefits using either rating scale or person trade-off elicitation methods. In some experiments, subjects were asked to rate the utility of the health benefits with attention also to the cost of achieving the benefits. In all experiments, at the end, subjects were shown a priority list based on their own utility judgments and were asked whether they wanted to move any of the health benefits up or down the list. Results. In all experiments, subjects wanted to give higher priority to treatments with higher benefit, even when they also had higher cost. They thus wanted to give less weight to high cost (which would, by itself, lead to lower ranking) and more weight to benefit than the weight implied by their own prior judgments. The desire for revision was reduced when subjects made their utility judgments after indicating whether the utility was above or below the midpoint of the scale (a manipulation previously found to reduce distortion). Conclusion. The desire to change cost-effectiveness rankings is in part a preference reversal phenomenon that occurs because people attend mainly to the benefit of health interventions as opposed to cost, when they examine the ranking. People should be wary of tinkering with priority lists by examining the lists themselves.