When a traditional Deep Deterministic Policy Gradient (DDPG) algorithm is used in mobile robot path planning, due to the limited observable environment of mobile robots, the training efficiency of the path planning model is low, and the convergence speed is slow. In this paper, Long Short-Term Memory (LSTM) is introduced into the DDPG network, the former and current states of the mobile robot are combined to determine the actions of the robot, and a Batch Norm layer is added after each layer of the Actor network. At the same time, the reward function is optimized to guide the mobile robot to move faster towards the target point. In order to improve the learning efficiency, different normalization methods are used to normalize the distance and angle between the mobile robot and the target point, which are used as the input of the DDPG network model. When the model outputs the next action of the mobile robot, mixed noise composed of Gaussian noise and Ornstein–Uhlenbeck (OU) noise is added. Finally, the simulation environment built by a ROS system and a Gazebo platform is used for experiments. The results show that the proposed algorithm can accelerate the convergence speed of DDPG, improve the generalization ability of the path planning model and improve the efficiency and success rate of mobile robot path planning.

Deep Deterministic Policy Gradient (DDPG) is a deep reinforcement learning algorithm that is widely used in the path planning of mobile robots. It solves the continuous action space problem and can ensure the continuity of mobile robot motion using the Actor-Critic framework, which has great potential in the field of mobile robot path planning. However, because the Critic network always selects the maximum Q value to evaluate the actions of mobile robot, there is the problem of inaccurate Q value estimation. In addition, DDPG adopts a random uniform sampling method, which can’t efficiently use the more important sample data, resulting in slow convergence speed during the training of the path planning model and easily falling into local optimum. In this paper, a dueling network is introduced based on DDPG to improve the estimation accuracy of the Q value, and the reward function is optimized to increase the immediate reward, to direct the mobile robot to move faster toward the target point. To further improve the efficiency of experience replay, a single experience pool is separated into two by comprehensively considering the influence of average reward and TD-error on the importance of samples, and a dynamic adaptive sampling mechanism is adopted to sample the two experience pools separately. Finally, experiments were carried out in the simulation environment created with the ROS system and the Gazebo platform. The results of the experiments show that the proposed path planning algorithm has a fast convergence speed and high stability, and the success rate can reach 100% and 93% in the environment without obstacles and with obstacles, respectively.