Understanding the Risks of Forecasting

A fundamental element of the successful deployment of operational teams is the ability to forecast future demand accurately. There is a raft of forecasting methods available to the forecaster ranging from the simple to the highly sophisticated, with many software packages able to identify the most appropriate method. One of the skills of the forecaster is the ability to strike a balance between finding a model that is good enough and one that overfits the data, especially if there is a demand from the business to seek forecasts with greater accuracy. However, the pursuit of better forecasts can sometimes be a fruitless exercise and Is it important for any forecaster to ask two questions: firstly, Is it of benefit to improve accuracy, and secondly, given the data available, can better forecasts be produced? The first question is a matter of identifying the effort required to produce better forecasts, be that through identifying a better model or collecting additional data to improve the existing model, and weighing it against the improvements in accuracy. The second question is a matter of understanding the limitations of the current forecasting model and, as importantly, the data used to produce the forecasts. If, for instance, the data contains a high level of randomness, even the most sophisticated model will be unable to produce highly accurate forecasts. This chapter describes an approach based on forecast errors to derive an approximation of the signal-to-noise ratio that can be used to understand the limitations of a given forecasting method on a given set of data and therefore provides the forecaster with the knowledge of whether there is any likely benefit from seeking further improvements. If the forecaster accepts the current model, the signal-to-noise ratio can provide an understanding of the risks associated with that model.