Enhanced Dynamic Expansion Planning Model Incorporating Q-Learning and Distributionally Robust Optimization for Resilient and Cost-Efficient Distribution Networks

The increasing integration of renewable energy-based distributed generation (DG) in modern distribution networks is essential for reducing reliance on fossil fuels. However, the unpredictability and intermittency of renewable sources such as wind and photovoltaic (PV) systems introduce significant challenges for distribution network planning. To address these challenges, this paper proposes a Q-learning-based Distributionally Robust Optimization (DRO) model for expansion planning of distribution networks and generation units. The proposed model incorporates energy storage systems (ESSs), renewable DG, substations, and distribution lines while considering uncertainties such as renewable generation variability, load fluctuations, and system contingencies. Through a dynamic decision-making process using Q-learning, the model adapts to changing network conditions to minimize the total system cost while maintaining reliability. The Latin Hypercube Sampling (LHS) method is employed to generate multi-scenario data, and piecewise linearization is used to reduce the computational complexity of the AC power flow equations. Numerical results demonstrate that the model significantly improves system reliability and economic efficiency under multiple uncertainty scenarios. The results also highlight the crucial role of the ESS in mitigating the variability of renewable energy and reducing the expected energy not supplied (EENS).