Renewable energy forecasting: A self-supervised learning-based transformer variant

Reliable and accurate renewable energy forecasting (REF) has substantial impact on society by helping with daily planning and mitigating the instability of power system. Providing a state-of-the-art (SOTA) solution for REF can power it to take on more sophisticated tasks and solve frontier problems to play a greater role in modern society. Aiming this goal, this work proposes a novel Transformer-based model, named Graph Patch Informer (GPI), for REF. Compared with existing REF models and mainstream Transformer, our model has three main characteristics: (1) Segment-wise self-attention is designed, which benefits Transformer by preserving the temporal information hidden in the continuous signals, (2) Graph Attention Networks with adaptive adjacent matrix is proposed to capture the inter-temporal dependencies automatically, (3) A new training strategy, that is, self-supervised pre-training followed by fine-tuning, is introduced to enhance the representation learning. To validate the performance of GPI, five experiments are conducted on four datasets covering solar radiation (SR), photovoltaic power (PVP), wind speed (WS) and wind power (WP). Experiments show that GPI goes beyond SOTA Autoformer by 23.67%–40.75% on MSE. It demonstrates that GPI can provide an effective solution for REF and have SOTA performance on SR, PVP, WS, WP forecasting tasks. Experiments also show that GPI can mitigate the adverse influences caused by missing values to a certain degree.