A Temporal Linear Network for Time Series Forecasting

Recent research has challenged the necessity of complex deep learning architectures for time series forecasting, demonstrating that simple linear models can often outperform sophisticated approaches. Building upon this insight, we introduce a novel architecture the Temporal Linear Net (TLN), thatextends the capabilities of linear models while maintaining interpretability and computational efficiency. TLN is designed to effectively capture both temporal and feature-wise dependenciesin multivariate time series data. Our approach is a variant of TSMixer that maintains strict linearity throughout its architecture. TSMixer removes activation functions, introduces specialized kernel initializations, and incorporates dilated convolutions to handle various time scales, while preserving the linear nature of the model. Unlike transformer-based models that may losetemporal information due to their permutation-invariant nature, TLN explicitly preserves and leverages the temporal structure of the input data. A key innovation of TLN is its ability to compute an equivalent linear model, offering a level of interpretability not found in more complex architectures such as TSMixer. This feature allows for seamless conversion between the full TLN model and its linear equivalent, facilitating both training flexibility and inference optimization. Importantly, we demonstrate that TLN outperforms standard linear regression models. This superior performanceis attributed to the unique training structure of the TLN and the inherent relationship of weights established during model construction, which can not be found in conventional regression approaches. Our findings suggest that TLN strikes a balance between the simplicity of linear models and the expressiveness needed for complex time series forecasting tasks. It offers improvedinterpretability compared to TSMixer while demonstrating greater resilience in multivariate cases than basic linear models and standard linear regression. This work contributes to the ongoingdiscussion about the trade-offs between model complexity and performance in time series analysis, and opens new possibilites for developing efficient and interpretable forecasting models that leverage the strengths of neural network architectures and linear models.