Practicability analysis of online deep reinforcement learning towards energy management strategy of 4WD-BEVs driven by dual-motor in-wheel motors

Deep reinforcement learning (DRL) has emerged as a promising approach for optimizing energy management strategies (EMS) in new energy vehicles. Nevertheless, existing studies typically focus on evaluating the performance of a particular algorithm, overlooking critical implementation details and lacking comprehensive analysis of their real-world applicability. In this paper, we mitigate this issue by conducting a thorough practicability analysis of DRL-based EMS methods. First, we theoretically analyze the benefits and limitations of existing DRL-based EMS approaches based on taxonomy towards their practicability. Subsequently, a novel EMS method that leverages model-based DRL algorithms that other researchers typically underestimate is proposed. The method newly introduces an uncertainty-aware model-based algorithm known as Probabilistic Ensembles with Trajectory Sampling (PETS) and is validated utilizing a four-wheel-drive (4WD) battery electric vehicle (BEV). After that, we conduct a comprehensive practicability analysis of three state-of-the-art DRL algorithms considering critical aspects for real-world deployment, e.g., hyperparameter sensitivity and algorithm transferability. The results demonstrate that even though the on-policy DRL achieves better asymptotic rewards and the off-policy DRL possesses better convergence, the proposed model-based DRL, PETS-based EMS, outperforms others regarding superior robustness and promising transferability across different extents of relevance between tasks. Besides, energy consumption results demonstrate that the model-based EMS can achieve a considerable 96.6% optimality compared to the baseline dynamic programming (DP). Thus, motivated by the challenges of applying DRL algorithms to real-world EMS, our systematic investigation and new insights contribute to advancing the practical employment of DRL-based EMS for BEVs.