An approach to evaluating energy-economy models

The increasing importance of models that stress the energy portion of the economy has created the demand for evaluation, by other than the model developer, to determine the strengths and weaknesses of a model with respect to the needs of the intended user. The literature describing how evaluations should be performed shows agreement on several basic components of the evaluation process, but at the same time is confusing because, although the same words may be used to describe these components, they often have different meanings. Although the models of interest are quantitative and the corresponding computer programs yield numerical results, published guidelines on model evaluation have lacked operationally defined checkpoints that can be used to summarize concisely and quantitatively the status of knowledge at a given time. Information related to desirable endpoints such as user requirements in terms of model outputs and associated maximum uncertainties, data sensitivities, etc., have customarily been expressed in more qualitative terms that are difficult to measure. A set of guidelines is developed for evaluating energy-economy models; the components of a thorough model evaluation are defined in an operational manner. Quantitative measures based on sensitivity and uncertainty analysis of results important to the user are included. Examples of the various components are drawn from the literature and from evaluation by Oak Ridge National Laboratory of the Long-Term Energy Analysis Program (LEAP) used by the Energy Information Administration for making projections to year 2020 included in that agency's 1978 Annual Report to Congress. Possible paths for future development of evaluation techniques are indicated, particularly those for which evaluation components must be more strictly defined after further practical experience is gained.