An Alternative Reinforcement Learning (ARL) control strategy for data center air-cooled HVAC systems

Energy efficiency of data center is of great concern globally due to their large amount of energy consumption and the foreseeable growth in the demand of digital services in the future. Advanced control strategies are needed to reduce energy consumption. However, the optimization of data center HVAC control is a challenge task due to the complexity of the thermal dynamic models of buildings and uncertainties associated with both server loads and outdoor temperature. In this paper, we propose a new control strategy called Alternately-Reinforcement Learning-control(ARL), which realizes the alternating control of RL and proportional, integral and derivative (PID) for optimizing the control of air-cooled HVAC systems in data centers. The control object of the ARL is the speed set of the compressor and the condensing fan, and the control goal is to minimize energy consumption while maintaining temperature stability. The applied RL algorithm is Deep deterministic policy gradient (DDPG) and Proximal policy optimization (PPO). We pre-train the ARL strategy in offline environment firstly and deploy it in real data center environment for online testing. The test results show that the ARL has significant advantages in maintaining temperature stability while reducing energy consumption, and the PPO algorithm performs better than the DDPG algorithm. Compared with the PID algorithm, the PPO algorithm can save energy by 5.27%, and the temperature control effect can be improved by 3.27%,which indicates the feasibility of the ARL for implementation in real data centers.