When the camera moves quickly and the image is blurred or the texture in the scene is missing, the Simultaneous Localization and Mapping (SLAM) algorithm based on point feature experiences difficulty tracking enough effective feature points, and the positioning accuracy and robustness are poor, and even may not work properly. For this problem, we propose a monocular visual odometry algorithm based on the point and line features and combining IMU measurement data. Based on this, an environmental-feature map with geometric information is constructed, and the IMU measurement data is incorporated to provide prior and scale information for the visual localization algorithm. Then, the initial pose estimation is obtained based on the motion estimation of the sparse image alignment, and the feature alignment is further performed to obtain the sub-pixel level feature correlation. Finally, more accurate poses and 3D landmarks are obtained by minimizing the re-projection errors of local map points and lines. The experimental results on EuRoC public datasets show that the proposed algorithm outperforms the Open Keyframe-based Visual-Inertial SLAM (OKVIS-mono) algorithm and Oriented FAST and Rotated BRIEF-SLAM (ORB-SLAM) algorithm, which demonstrates the accuracy and speed of the algorithm.
Teaching robots to learn through human demonstrations is a natural and direct method, and virtual reality technology is an effective way to achieve fast and realistic demonstrations. In this paper, we construct a virtual reality demonstration system that uses virtual reality equipment for assembly activities demonstration, and using the motion data of the virtual demonstration system, the human demonstration is deduced into an activity sequence that can be performed by the robot. Through experimentation, the virtual reality demonstration system in this paper can achieve a 95% correct rate of activity recognition. We also created a simulated ur5 robotic arm grasping system to reproduce the inferred activity sequence.
30 July 2025 in Arabian Journal for Science and Engineering
The automation of assembly operations with industrial robots is pivotal in modern manufacturing, particularly for multispecies, low-volume, and customized production. Traditional programing methods are time-consuming and lack adaptability to complex, variable environments. Reinforcement learning-based assembly tasks have shown success in simulation environments, but face challenges like the simulation-to-reality gap and safety concerns when transferred to real-world applications. This article addresses these challenges by proposing a low-cost, image-segmentation-driven deep reinforcement learning strategy tailored for insertion tasks, such as the assembly of peg-in-hole components in satellite manufacturing, which involve extensive contact interactions. Our approach integrates visual and forces feedback into a prior dueling deep Q-network for insertion skill learning, enabling precise alignment of components. To bridge the simulation-to-reality gap, we transform the raw image input space into a canonical space based on image segmentation. Specifically, we employ a segmentation model based on U-net, pretrained in simulation and fine-tuned with real-world data, significantly reducing the need for labor-intensive real image segment labels. To handle the frequent contact inherent in peg-in-hole tasks, we integrated safety protections and impedance control into the training process, providing active compliance and reducing the risk of assembly failures. Our approach was evaluated in both simulated and real robotic environments, demonstrating robust performance in handling camera position errors and varying ambient light intensities and different lighting colors. Finally, the algorithm was validated in a real satellite assembly scenario, achieving a success rate of 15 out of 20 tests.
19 September 2024 in Arabian Journal for Science and Engineering
Nowadays, more and more applications require precise and quickly 3D recognition, such as augmented reality and robot navigation. In recent years, model-based methods can get accurate object or scene recognition, but it takes a lot of time to reconstruct the model. Therefore, we propose a fast 3D reconstruction method based on ordered images for robust and accurate 3D recognition. The proposed algorithm consists of two parts, the offline processing stage, and the online processing stage. First, in the offline processing stage, the sparse point cloud model of the scene or object is reconstructed based on the sequential images, optimized using the BA algorithm based on the local correlation frame, and then the local descriptor of the resulting model points is stored. Secondly, in the online processing stage, for each image frame of the camera video, a matching relationship between the stored point cloud and the 2D feature points on the image frame is established, based on which the pose of the camera can be solved accurately.