Predictive reinforcement learning: map-less navigation method for mobile robot

The application of reinforcement learning in mobile robotics faces the challenges of real-world physical environments, in contrast to playground setups like video games. In a mobile robot motion control, it is not always possible to perform episodes of pre-training in large amounts due to time, resource limitations or other concerns. Control methods that rely on a prior explicit map may be impractical or even impossible to use for new dynamic environments. In this paper, we present a method of local navigation approach for driving a robot to a desired position without relying on an explicit map of the environment. Only the laser scan measurements were used to determine the obstacles. We focus in this work on online methods of reinforcement learning which do not require running the robot in full episodes until success or failure. However, the price for such an online capability is that some model knowledge about the environment has to be utilized. Here, we propose an algorithm called stacked Q-learning, which unifies aspects of standard reinforcement learning techniques with model-based predictive agents. We compare this algorithm to a classical model predictive controller. The comparison focuses on the accumulated cost of parking the robot avoiding obstacles. The results look promising as the stacked Q-learning beat its counterpart, model predictive control, yet being of the same computational complexity. The suggested agent design of stacked Q-learning can thus be taken as a foundation for a class of predictive reinforcement learning methods.