In order to achieve collision-free path planning in complex environment, Munchausen deep Q-learning network (M-DQN) is applied to mobile robot to learn the best decision. On the basis of Soft-DQN, M-DQN adds the scaled log-policy to the immediate reward. The method allows agent to do more exploration. However, the M-DQN algorithm has the problem of slow convergence. A new and improved M-DQN algorithm (DM-DQN) is proposed in the paper to address the problem. First, its network structure was improved on the basis of M-DQN by decomposing the network structure into a value function and an advantage function, thus decoupling action selection and action evaluation and speeding up its convergence, giving it better generalization performance and enabling it to learn the best decision faster. Second, to address the problem of the robot’s trajectory being too close to the edge of the obstacle, a method of using an artificial potential field to set a reward function is proposed to drive the robot’s trajectory away from the vicinity of the obstacle. The result of simulation experiment shows that the method learns more efficiently and converges faster than DQN, Dueling DQN and M-DQN in both static and dynamic environments, and is able to plan collision-free paths away from obstacles.
In order to solve the problem of difficult and low precision in the detection of safety helmet wearing in the complex pose of construction worker, a detection method of safety helmet wearing based on pose estimation is proposed. In the pose estimation model of OpenPose, the residual network optimized feature extraction is introduced to obtain the skeletal point information of the construction worker, and then the pose of the construction worker is estimated based on the skeletal point information, three‐point localization method is proposed for the front and back pose, and skin colour detection method is proposed for the side pose, and then to determine the head region. The YOLO v4 is used to detect the safety helmet region, and then the construction worker's safety helmet wearing is judged according to whether the head region intersects the safety helmet region or not. Experimental results show that the detection accuracy of the method is higher than other methods, and the adaptability to the environment is stronger.
14 June 2023 in Journal of Circuits, Systems and Computers
In the field of robot path planning, Deep Reinforcement Learning (DRL) has demonstrated considerable potential as a cutting-edge artificial intelligence technology. However, the effective utilization of representation learning in path planning tasks, which is pivotal for successful DRL performance, has remained elusive. This challenge arises from the predominant use of compact vectors derived directly from low-level sensors as the state representation in the task. Learning meaningful representations on such low-level states often proves to be challenging. To address this issue, a novel approach named Contrastive Learning Regularized Feature-Enhanced Actor-Critic (CFEAC) is proposed in this paper. This method adopts a contrastive learning perspective to handle features in neural networks and incorporates cross-layer connections and deep networks to achieve feature enhancement. In a constructed 3D point cloud simulation environment, the CFEAC algorithm outperforms DDPG, TD3, SAC, and SALE algorithms in terms of higher cumulative reward and lower collision rates. Experimental results validate that this approach exhibits superior path planning performance in complex static and dynamic scenarios.