Asymmetry and Interdependence when Evaluating U.S. Energy Information Agency Forecasts

We evaluate US Energy Information Agencies (EIA) forecasts of the world petroleum market, emphasising the importance of taking a multivariate perspective, considering asymmetric loss and allowing for time-variation. Forecasts for total demand, total supply, total stock withdrawals and the oil prices are biased, with biases that change over time and differ across variables. A loss function that takes into account asymmetry and interdependence can rationalise these biases. The implied asymmetric loss gives less weight to under-prediction of both demand and supply, while for oil prices, we document significant regime changes in the implied loss due to asymmetry. The EIA forecasts dominate a simple random walk benchmark when evaluated using symmetric and independent loss in the form of MSE statistical criteria. Yet, when allowing for asymmetry and interdependence that rationalize the EIA forecasts, the performance of the EIA forecasts worsens and is comparable to the random walk benchmark.