Selecting a Model for Forecasting

Jennifer L. Castle, Jurgen A. Doornik and David F. Hendry We investigate the role of the significance level when selecting models for forecasting as it con-trols both the null retention frequency and the probability of retaining relevant variables when using binary decisions to retain or drop variables. Analysis identifies the best selection significance level in a bivariate model when there are location shifts at or near the forecast origin. The trade-off for select¬ing variables in forecasting models in a stationary world, namely that variables should be retained if their non-centralities exceed 1, applies in the wide-sense non-stationary settings with structural breaks examined here. The results confirm the optimality of the Akaike Information Criterion for forecasting in completely different settings than initially derived. An empirical illustration forecast¬ing UK inflation demonstrates the applicability of the analytics. Simulation then explores the choice of selection significance level for 1-step ahead forecasts in larger models when there are unknown lo¬cation shifts present under a range of alternative scenarios, using the multipath tree search algorithm, Autometrics (Doornik, 2009), varying the target significance level for the selection of regressors. The costs of model selection are shown to be small. The results provide support for model selection at looser than conventional settings, albeit with many additional features explaining the forecast perfor¬mance, with the caveat that retaining irrelevant variables that are subject to location shifts can worsen forecast performance.