Improving the Consistency of Conditional Probability Assessments for Forecasting and Decision Making

"Public agencies are very keen on amassing statistics---they collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But what you must never forget is that every one of those figures comes in the first instance from the village watchman, who just puts down what he damn pleases." (Sir Josiah Stamp) The assessment of the conditional probabilities of events is useful and needed for forecasting, planning, and decision making. In this paper the difficulties associated with the assessment of these conditional probabilities are examined. The necessary and sufficient conditions that the elicited information on conditional probabilities must satisfy are evaluated against actual assessments in several different controlled settings. A high frequency of implicit violations of the probability calculus was observed. The consistency of the assessments is affected by the causal/diagnostic and positive/negative relationships of the events. Use of a judgmental aid in the form of a joint probability table reduces the number of inconsistent responses significantly. Using the probability axioms, it is also shown that only the first order conditional probabilities need be assessed, as higher order probabilities are robust to the unconditional and first order conditional assessments.