Modeling the dynamic allocation problem of multi-service storage system with strategy learning

Gas storage, especially underground gas storage (UGS), plays a particularly important role in coping with supply disruptions. UGSs provide flexible services, e.g. firm storage service (FSS), interruptible storage service (ISS), and price hedging service (PHS), to their customers to harmonize the fluctuations in gas demand. At the beginning of each period, UGS has to allocate the overall capacity for different services for market auction. Allocation of available capacity is generally based on historical data and hands-on experience, which is operationally and economically inefficient. To address the problem, the paper developed a reinforcement learning model that captures the stochastic and dynamic features of the gas allocation problem. We study the optimal gas dispatching strategy under conditions where uncertain market states (e.g., demand load and price) exist in a continuous state space. The decision maker can improve their strategy by evaluating the expected rewards of storage actions in response to specific market states (or “learning” in the environment). In a 360-day numerical experiment, the model and algorithm demonstrate high efficiency in solving the problem. Meanwhile, several managerial implications regarding working gas volume (WGV) and agreed capacity are proposed to help improve the operational efficiency of UGS.