OLS-Regression Forecasting Confidence Intervals Capture Rates: Precision Profiling in the Forecasting Model Selection Process

Forecasting creates projections into an uncertain future. To understand the decision-making implications of the forecast, confidence intervals [CIs] are required. This seems simple enough. However, considering the details of the computations that underlie generating forecasting CIs, one encounters a practical disconnect that can create a decision-making paradox. In the OLS 2-parameter linear regression [OLSR] case, there are two forecasting Models that are often employed- {The Uni-variate Time Series & The Standard two-variable Regression Y-X}. Further, for each of these two Models individually, there are three (1-FPE[α]) %CIs Versions- {Excel, Fixed Effects & Random Effects} each of which is oriented around the same OLSR-forecast value. Given this component configuration, a paradox emerges because each of the forecasting models, {TS or Y-X}, individually produces a forecast with a markedly difference precision profile over the three CI-Versions. In our experience, this is paradoxical as forecasters assume that as the forecasts are the same in each model-class, their Capture Rate—the percentage of time that the actual future values are IN the CIs—should also be the same. To address this seeming paradox, we develop, detail, and illustrate a two-stage OLSR Decomposition and Screening protocol, termed- the [D&S-Triage] protocol that has the following components- (i) Stage A- decomposition of the Model & Version factor-sets to better understand the implications of their Precision differences, and (ii) Stage B- focusing on inferentially significant forecasting model components, create a multilevel quality-algorithm to identify a forecasting model-set that addresses the Quality of the Capture Rate that are the best in their class.