Optimal operations planning of electric autonomous vehicles via asynchronous learning in ride-hailing systems

Ride-hailing systems with electric autonomous vehicles are recognized as a next-generation development to ease congestion, reduce costs and carbon emissions. In this paper, we consider the operation planning problem involving vehicle dispatching, relocation, and recharging decisions. We model the problem as a Markov Decision Process (MDP) to generate the optimal policy that maximizes the total profits. We propose a flexible policy to provide optimal actions according to the reward considering future requests and vehicle availability. We show that our model outperforms the predetermined rules on improving profits. To handle the curse-of-dimensionality caused by the large scale of state space and uncertainty, we develop an asynchronous learning method to solve the problem by approximating the value function. We first draw the samples of exogenous information and update the quality of approximations based on the quality of decisions, then approximate the exact cost-to-go value function by solving an approximation subproblem efficiently given the state at each period. Two variant algorithms are presented for cases with scarce and sufficient information. We also incorporate the state aggregation and post-decision analysis to further improve computational efficiency. We use a set of shared actual data from Didi platform to verify the proposed model in numerical experiments. To conclude, we extract managerial insights that suggest important guidelines for the ride-hailing operations planning problem.