Model credibility and independent evaluation--Three case studies

There has been a rapidly expanding use of complex models as aids to policy making. However, it has become increasingly clear that there is little understanding of the capabilities of these models, and thus their credibility and their usefulness have been greatly diminished. To help correct this situation efforts have been conducted to improve some models' usefulness through intensive independent evaluations. The state-of-the-art of the conduct of such evaluations is presented in this article by way of a review of some of the existing literature. In addition, first presented here are three case studies of in-depth model evaluations, particularly concentrating on the lessons learned with regard to the uncovering of assessment issues. A final section contains the lessons that have been learned with regard to predicting the usefulness of specific model assessments, so that the value of future assessments can be maximized through the intelligent selection of candidate models and assessment strategies.