Global optimization guided energy management strategy for hybrid electric vehicles based on generative adversarial network embedded reinforcement learning

The learning-based energy management strategy (EMS) for hybrid electric vehicles (HEVs) are predominantly driven by deep reinforcement learning (DRL). However, DRL requires substantial time and computational resources for aimless trial and error, despite its application in problems like EMS that can yield global optimal solutions. This paper draws on the framework of Generative Adversarial Imitation Learning (GAIL), utilizing a discriminator network as a conduit for the EMS transitioning from Proximal Policy Optimization (PPO) to the demonstrations based on Dynamic Programming (DP), thereby enhancing the training efficiency and optimality. Initially, expert state-action data is extracted from the offline DP solution. Subsequently, an online PPO state-action generator for EMS is constructed, which trains the discriminator alongside the expert data. The discriminator identifies all policies that deviate from the demonstrations, while PPO generator aims at deceiving the discriminator, thus fostering a mutually beneficial adversarial relationship. Simulation results indicate that the proposed method shortens training time by 68.4 % compared to the original PPO, achieving fuel economy exceeding 95 % under varying initial state of charge (SoC) of the power battery.