Predictive Evaluation of Econometric Forecasting Models in Commodity Futures Markets

The predictive accuracy of various econometric models, including random walks, vector-autoregressive and vector-error-correction models, are investigated using daily futures prices of four commodities (the S&P 500 index, treasury bonds, gold, and crude oil). All models are estimated using a rolling-window approach, and evaluated by both in-sample and out-of-sample performance measures. The criteria considered include system criteria, where we evaluate multiequation forecasting models, and univariate forecast-accuracy criteria. The five univariate criteria are root mean square error (RMSE), mean absolute deviation (MAD), mean absolute percentage error (MAPE), confusion matrix (CM), and confusion rate (CR). The five system criteria used include the trace of second-moment matrix of the forecast-errors matrix (TMSE), the trace of second-moment matrix of percentage-forecast errors (TMAPE), the generalized forecast-error second-moment matrix (GFESM), and a trading-rule profit criterion (TPC) based on a maximum-spread trading strategy. An in-sample criterion, the mean Schwarz information criteria (MSIC), is also computed. Our results suggest that error-correction models perform better in shorter forecast horizons, when models are compared based on quadratic loss measures and confusion matrices. However, the error-correction models which we consider perform better at all forecast horizons (one to five steps ahead) when models are compared based on a profit-maximization loss function. Further, our error-correction model, where the error-correction term is constructed according to a cost-of-carry equilibrium condition, outperforms our alternative error-correction model, which uses the price spreads as the error-correction term.