Incorporating key features from structured and unstructured data for enhanced carbon trading price forecasting with interpretability analysis

Accurate prediction of carbon trading prices is crucial for societal progression, prompting a shift toward harnessing unstructured data sources to improve forecasting accuracy and interpretability. Despite this trend, the utilization of unstructured data like the Baidu search index remains inadequate. Additionally, conventional multivariate forecasting methods face challenges in balancing accuracy with interpretability. To address these limitations, we propose an innovative methodology that integrates both structured and unstructured data streams to enhance carbon trading price forecasting, where an interpretability analysis framework is used to explore the influencing factors of predictive models. Firstly, we conduct feature analysis focusing on structured data including historical carbon price data, key supply-demand factors, and unstructured data sourced from Baidu Index keywords. Besides, we simplify the data pattern through statistical analysis to reduce dimensionality, and refine it further via CEEMDAN decomposition coupled with wavelet threshold denoising for a cleaner dataset. Subsequently, we establish a support vector regression model leveraging the multi-dimensional and small-sample modeling, suitable for this multi-source and mixed structures data. Finally, we introduce a random interference mechanism based on model features, then construct a mathematical framework for the differences in model accuracy loss, providing insights into the model's decision-making mechanisms. This work demonstrates that the data cleaning stage can provide more predictive data for subsequent prediction models, and the proposed model is validated through rigorous experimental design for its superiority in model comparison analysis, multi-step prediction, and interpretability analysis of selected features.