Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network

Fuel cell hybrid vehicle (FCHV) is an effective way to improve emissions and fuel economy. Energy management strategy (EMS), as a core assignment of FCHV, its most significant challenge is to achieve a sensible trade-off between system degradation and hydrogen consumption with less computational cost under diverse environments. Although most strategies can employ finite-density discretization to approach the optimum solutions, it brings poor performance. Herein, an advanced dueling-double-deep Q-network (D3QN) EMS based on a deep reinforcement learning framework is developed to solve the challenges, which can process higher-dimensional space and output dominant actions for an agent to obtain higher cumulative rewards. By adopting a dueling neural network, a better policy is acquired via generalizing learning across actions in the presence of similar value-actions. Moreover, an evaluation mechanism of Allowable Approach Punishment is introduced into rewards to mitigate system degradation. Simulation experiments reveal the prominence of the D3QN algorithm. Results demonstrate D3QN achieves less hydrogen consumption and retards fuel cell degradation on the premise of meeting power balance. Furthermore, adaptability and robustness are verified. Outcomes show D3QN can not only adapt to complex surroundings but also overcome noise and instability. Finally, real-time applicability is validated via the HIL test-bench.