Multi-period, multi-timescale stochastic optimization model for simultaneous capacity investment and energy management decisions for hybrid Micro-Grids with green hydrogen production under uncertainty

Given the steep rises in renewable energy's proportion in the global energy mix expected over several decades, a systematic way to plan the long-term deployment is needed. The main challenges are how to handle the significant uncertainties in technologies and market dynamics over a large time horizon. The problem is further complicated by the fast-timescale volatility of renewable energy sources, potentially causing grid instability and unfulfilled demands. As a remedy, energy storage and power-to-hydrogen systems are considered in conjunction with energy management system but doing so raises the complexity of the planning problem further. In this work, the long-term capacity planning for a hybrid microgrid (HM) system is formulated as a multi-period stochastic decision problem that considers uncertainties occurring at multiple timescales. Long-term capacity decisions are inherently linked with energy dispatch and storage decisions occurring at fast-timescale and it is best to solve for them simultaneously. However, the computational demand for solving it becomes quickly intractable with problem size. To this end, we propose to develop a Markov decision process (MDP) formulation of the problem and use simulation-based reinforcement learning for multi-period capacity investments of the planning horizon. The MDP includes the policies used for dispatch and storage operation, which are represented by linear programming as a part of the simulation model. The effectiveness of our proposed method is demonstrated with a case study, where decisions over multiple decades are considered along with various uncertainties of multi-timescales. Economic and environmental assessments are performed, providing valuable guidelines for government's energy policy.