3 March 2020 in IEEE Access
In order to solve the problem of slow convergence rate and long planned path when the robot plans a path in unknown environment by using Q-learning algorithm, we propose an Experience-Memory Q-Learning (EMQL) algorithm based on the continuous update of the shortest distance from the current state node to the start point. The autonomous learning ability of the robot is enhanced by the different role assignment of two tables in the proposed algorithm. EM table with (m * 1) dimension is designed to record the distance information, reflecting the learning process of the robot. Q table is adopted as an auxiliary guidance for the experience transfer strategy and experience reuse strategy, and these strategies enable the robot accomplish the task even if the destination is changed or the path is blocked. Further, the learning efficiency of the robot in the EMQL algorithm is improved by the dual reward mechanism consisting of static reward and dynamic reward. The static reward is designed to prevent the robot from exploring a state node excessively. The dynamic reward is responsible for helping the robot avoid searching blindly in unknown environment. We verify the effectiveness of the proposed algorithm on the grid maps and road network maps. The comparison about planning time, iteration times and path length show that the performance of the EMQL algorithm is superior to Q-learning algorithm in convergence speed and optimization ability. Additionally, the practicability of the proposed algorithm is further validated in a real-world experiment using the Turtlebot3 burger robot.
10 August 2022 in Applied Soft Computing
15 January 2025 in IEEE Transactions on Automation Science and Engineering
The multi-robot system that combines flexibility and autonomy has unique advantages in many scenarios. In unknown environment, the threat posed by the environmental uncertainty to the path planning process cannot be ignored. The robots not only need to avoid obstacles, but also need to maintain communication with other robots to ensure timely negotiation. Inspired by the division of labor in biological community, this paper proposes robot behavior scheduling algorithm for multi-robot path planning in unknown environment with communication constraints. First, to avoid the occurrence of oscillation caused by a single force determining the robot behavior, the constituent elements of robot behavior are described in the form of multiple forces. Then, various robot behaviors are instantiated through a diverse combination of these elements and parameters. Finally, two complementary robot behavior scheduling methods are designed. The experiential potential field jointly constructed by robots is proposed to guide the robots to juggle avoidance, communication and planning task for the first time. The effectiveness of these behavior scheduling methods is verified in theory. The simulation experiment results based on real sensor parameters show that a small amount of robot detours can make the multi-robot system fully connected for 70% of the entire path planning process. Note to Practitioners—The motivation of this article is to avoid communication interruptions among robots when planning paths in unknown environment. When the robots perform the complex tasks, communication interruptions will lead to the inability of sharing environmental maps and allocation plans. The lack of such information may delay the task progress and even cause greater loss. This article focuses on the path planning demands in unknown environment and considers the automatic maintenance and reconstruction of the communication links among robots. Based on the analyses of the motivation in reaching the target point, avoiding obstacles, and maintaining communication and etc., we first model the constituent elements of robot behavior. Based on the designed robot behavior, the robot behavior scheduling algorithm is proposed. The experiment results demonstrate that the effectiveness of the proposed algorithm is not affected by the changes in task scenarios, signal loss conditions and the number of robots. The experiments on Turtlebot3 burger robots further validates the practicality of the proposed algorithm. Additionally, the robot behavior scheduling algorithm has scalability and can enable the robots to complete more complex tasks by enriching the types of robot behaviors and scheduling mechanisms.