Evaluating Individual and Mean Non-Replicable Forecasts

Macroeconomic forecasts are often based on the interaction between econometric models and experts. A forecast that is based only on an econometric model is replicable and may be unbiased, whereas a forecast that is not based only on an econometric model, but also incorporates expert intuition, is non-replicable and is typically biased. In this paper we propose a methodology to analyze the qualities of individual and means of non-replicable forecasts. One part of the methodology seeks to retrieve a replicable component from the non-replicable forecasts, and compares this component against the actual data. A second part modifies the estimation routine due to the assumption that the difference between a replicable and a non-replicable forecast involves measurement error. An empirical example to forecast economic fundamentals for Taiwan shows the relevance of the methodological approach using both individuals and mean forecasts.