Impact of Model Performance History Information on User's Confidence in Decision Models: An Experimental Examination

Effective decision support systems (DSS) must supply decision- makers with information that allows them to make correct judgments. Unfortunately, human intuitive judgments are subject to a number of biases. Among the judgments that a user of a DSS must make is the selection of an appropriate model. When a decision maker is presented with a history of a model's usage and frequency of success during that usage, the decision maker must judge how confident he/she is in the output that comes from that model. We show, in a laboratory setting using seventy-five student subjects and forty- eight managers, that decision makers can be manipulated into irrational confidence levels. In a corporate setting, over- and under-confidence will result in either overreliance on unreliable models or in a failure to take advantage of a useful tool.