6D pose estimation is a common and important task in industry. Obtaining the 6D pose of objects is the basis for many other functions such as bin picking, autopilot, etc. Therefore, many corresponding studies have been made in order to improve the accuracy and enlarge the range of application of various approaches. After several years of development, the methods of 6D pose estimation have been enriched and improved. Although some predecessors have analyzed the methods and summarized them in detailed, there have been many new breakthroughs in recent years. To understand 6D pose estimation better, this paper will make a new and more detailed review of 6D pose estimation. We divided these methods into two approaches: Learning-based approaches and non-learning-based approaches, including 2D-information-based approach and 3D-information-based approach. Additionally, we introduce the challenges that exist in 6D pose estimation. Finally, we compare the performance of different methods qualitatively and discuss the future development trends of the 6D pose estimation.
20 December 2018 in IEEE Transactions on Industrial Electronics
Conventional moment-based visual servoing methods suffer from several problems in industrial applications due to the utilization of high-order image moments. In this paper, we analyze the shortcomings of the moment-based visual servoing from the viewpoint of practical industrial applications, and propose a novel moment-based 2 1/2D visual servoing method for grasping textureless planar parts. We use hybrid visual features that combine image moments with 3D rotation in the Cartesian space to control the robot motion. Instead of applying high-order image moments, we use rotation features, which provide a decoupled interaction matrix that is full rank and with no local minimum in the control scheme. Furthermore, to estimate the relative rotation of the textureless part in real time, a new estimation method based on a cross-correlation analysis is proposed. The proposed visual servoing method provides a better motion control and 3D trajectory of the robot arm and remains stable in the workspace. Experimental results demonstrated the effectiveness of the method.
Laser stripe center extraction is a critical step for a structured-light system, which plays an important role in industrial fields. However, existing center extraction methods are mainly for simple objects, where the stripe contours are relatively clear and no fake stripe areas caused by surface texture or surface reflection disturb the extraction of the real stripe area. To develop center extraction methods for complex objects, this paper proposed a new laser stripe extraction method based on the features of the laser stripe. After extracting seed points of the laser stripe using an extreme value method and connected component analysis, we can determine the stripe directions from the seed points and iteratively grow the centerline of the laser stripe. Finally, a gray-gravity method is used to extract the accurate center points of the laser stripe based on the obtained centerline. Experiments on 2D image laser stripe extraction and real 3D reconstruction application verified the robustness and accuracy of the proposed method.
Pose estimation is an essential technology for industrial robots to perform precise gripping and assembly. The state-of-the-art deep learning-based approach uses an indirect strategy, i.e., first finding local correspondence between the 2-D image and 3-D model, and then using the perspective-n-point and RANSAC methods to calculate the poses of ordinary objects. However, the metal parts in industry are reflective and textureless, making it difficult to identify distinguishable point features to establish 2-D–3-D correspondences. To address this problem, in this article, we propose a novel deep learning based two-stage method for pose estimation of reflective textureless metal parts, which accurately estimates the target pose using monocular red green blue (RGB) images. Since contours play an important role in both keypoints prediction and pose estimation stages, our method is named ContourPose. First, an additional contour decoder is adopted to implicitly constrain the keypoints prediction in the former stage, which improves the accuracy of the keypoints prediction. Then, the predicted contour of the previous stage is taken as geometric prior that is used to iteratively solve for the optimal pose. Experiments indicate that the proposed approach for reflective textureless metal parts has a significant improvement over the state-of-the-art approaches.
8 April 2022 in IEEE Transactions on Industrial Informatics
TheSixdegrees of freedom 6DoF pose estimation of texture-less objects provides a spatial understanding of industrial scenes and is the basis for accurate object manipulation. Recent studies have shown that the introduction of known model information and initial poses helps CNNs-based methods achieve better performance in complex scenes. However, the mapping from image space to pose space learned by neural networks is dimensionally lifting. Due to the lack of depth information, it is difficult for neural networks to capture local clues on texture-less surfaces and directly regress the relative 3-D translation and 3-D rotation. Instead, we propose a novel framework named REG-Net, which transforms the 6DoF pose estimation task into a 2-D keypoint registration problem. The proposed network first encodes regional prior information using multi-representation, utilizes the globally-consistent offset attention module to align 2-D keypoint features in a long range, and then estimates offsets and potential regions of keypoints. The proposed regional PnP simultaneously adjusts the keypoint locations in a short range and outputs the pose. This framework compresses the learning space of the network from 3-D to 2-D. Extensive experiments on two benchmark datasets demonstrate the robustness and accuracy of REG-Net. We further demonstrate the effectiveness of REG-Net in the reflective industrial part grasping applications.