Real-time online charging control of electric vehicle charging station based on a multi-agent deep reinforcement learning

This paper proposes a multi-agent deep reinforcement learning-based charging scheduling strategy for electric vehicle (EV) charging stations, aiming to solve the problem of real-time online charging control of multiple EVs within a single charging station in an uncertain charging environment with random EV arrivals and departures. The proposed approach endeavors to maximize the benefits of EV drivers and charging station operators. First, a coordinated control framework for EV charging in the coupled transportation electrification system is constructed, and the Markov decision process is leveraged to describe the charging scheduling process of a single EV. The charging scheduling objective considers the charging station revenues, the overload penalty of charging station, EV drivers' charging comfort in the charging area, insufficient charging penalty in the charging area, and the waiting penalty in the waiting area. Second, a multi-agent deep reinforcement learning algorithm based on the centralized training with decentralized execution framework is developed. The algorithm utilizes an attention network to interact with the agents' observations and embeds an action mask layer to filter invalid actions. The charger serves as an agent that makes action decisions about charging power at each time slot. Finally, we utilize actual charging station operating data in Xi'an, China, to validate the effectiveness of the proposed approach in improving the overall benefits of charging stations and the scalability of the algorithm.