Identification and Correction of Abnormal, Incomplete Power Load Data in Electricity Spot Market Databases

The development of electricity spot markets necessitates more refined and accurate load forecasting capabilities to enable precise dispatch control and the creation of new trading products. Accurate load forecasting relies on high-quality historical load data, with complete load data serving as the cornerstone for both forecasting and transactions in electricity spot markets. However, historical load data at the distribution network or user level often suffers from anomalies and missing values. Data-driven methods have been widely adopted for anomaly detection due to their independence from prior expert knowledge and precise physical models. Nevertheless, single network architectures struggle to adapt to the diverse load characteristics of distribution networks or users, hindering the effective capture of anomaly patterns. This paper proposes a PLS-VAE-BiLSTM-based method for anomaly identification and correction in load data by combining the strengths of Variational Autoencoders (VAE) and Bidirectional Long Short-Term Memory Networks (BiLSTM). This method begins with data preprocessing, including normalization and preliminary missing value imputation based on Partial Least Squares (PLS). Subsequently, a hybrid VAE-BiLSTM model is constructed and trained on a loaded dataset incorporating influencing factors to learn the relationships between different data features. Anomalies are identified and corrected by calculating the deviation between the model’s reconstructed values and the actual values. Finally, validation on both public and private datasets demonstrates that the PLS-VAE-BiLSTM model achieves average performance metrics of 98.44% precision, 94% recall rate, and 96.05% F1 score. Compared with VAE-LSTM, PSO-PFCM, and WTRR models, the proposed method exhibits superior overall anomaly detection performance.