Real-time out-of-step prediction control to prevent emerging blackouts in power systems: A reinforcement learning approach

Blackouts impose undesired economical impacts and integrity issues on electric power systems. Early out-of-step prediction of generating units is of paramount importance for blackout prevention and power management. This work is concerned with the design of novel real-time mechanisms based on reinforcement learning for the early out-of-step prediction in small-scale and large-scale power systems while mitigating the rate of false and miss alarms. These mechanisms are enabled by formulating the out-of-step prediction problem as a partially observable Markov decision process, for which a reward shaping strategy is devised based upon deep Q-networks to support the learning process of the agent. The proposed prediction mechanisms are real-time and capable of dealing with the dynamic changes of loads. Various scenarios in the form of three separate experiments are simulated on Kundur’s two-area and IEEE 39-bus systems. The attained results verify the effectiveness of the proposed mechanisms in early out-of-step prediction when the received observations by the agent are noisy and the active power of loads is subject to dynamical changes.