22 January 2021 in IEEE Transactions on Circuits and Systems for Video Technology
Due to the lack of distinctiveness to the background, robust monocular 6DOF pose tracking of less-distinct objects remains an important open problem. In this paper, we firstly analyze the object distinctiveness in the tracking process. Then, we propose a novel contour-part model based robust monocular pose tracking method for less-distinct objects. This paper uses the traditional contour feature for pose tracking in a novel strategy called contour part model. First, the contour part model is built by segmenting the projected contour rendered from the 3D model into contour segments of a certain length adaptively according to the Shi-Tomasi cornerness scores. Then, the correspondence is detected in the input image for each contour part by gradient orientation based template matching. Finally, pose tracking is achieved by solving the PnP problem. Experiment results on semi-synthetic and real images show that the proposed method performs better than the existing methods when pose tracking less-distinct objects and shows great robustness toward interference. Additionally, we combine the edge feature and the regional feature in a simple strategy. The overall performance of the fused method is boosted according to the experimental results on the RBOT dataset. The fact shows that the fusion of the edge and regional features has great potential for improving tracking performance.
11 October 2023 in Visual Intelligence
Monocular object pose tracking has been a key technology in autonomous rendezvous of two moving platforms. However, rapid relative motion between platforms causes large interframe pose shifts, which leads to pose tracking failure. Based on the derivation of the region-based pose tracking method and the theory of rigid body kinematics, we put forward that the stability of the color segmentation model and linearization in pose optimization are the key to region-based monocular object pose tracking. A reliable metric named VoI is designed to measure interframe pose shifts, based on which we argue that motion continuity recovery is a promising way to tackle the translation-dominant large pose shift issue. Then, a 2D tracking method is adopted to bridge the interframe motion continuity gap. For texture-rich objects, the motion continuity can be recovered through localized region-based pose transferring, which is performed by solving a PnP (Perspective-n-Point) problem within the tracked 2D bounding boxes of two adjacent frames. Moreover, for texture-less objects, a direct translation approach is introduced to estimate an intermediate pose of the frame. Finally, a region-based pose refinement is exploited to obtain the final tracked pose. Experimental results on synthetic and real image sequences indicate that the proposed method achieves superior performance to state-of-the-art methods in tracking objects with large pose shifts.
3 September 2024 in IEEE Transactions on Instrumentation and Measurement
For infrared target segmentation, Convolutional Neural Networks based (CNN-based) methods have largely improved the segmentation accuracy. Previous CNN-based methods typically improved the segmentation accuracy by enhancing and aggregating multi-level features. But the existence of semantic gap between features from different semantic levels limits the improvement of feature aggregation. To address this problem and improve the segmentation accuracy of infrared targets with tiny sizes and irregular shapes, the Multi-Attention Shape Adaptive Network (MASANet) is proposed in this paper. Specifically, a Spatially Feature Fusion Dense-nested Unet (SFFDUnet) is designed as the feature extraction and aggregation backbone. Based on SFFDUnet, the Bi-directional Shape Adaptive Attention with Channel-wise Shape Enhance (BSAA-CSE) block is proposed as a multi-attention mechanism to reduce the semantic gap between features. Moreover, the multi-scale multi-branch deep supervision is introduced to stabilize the training process and enable model pruning. In addition, a new infrared target dataset CITY-IRTD is also constructed to evaluate the performance of our method. Experiments demonstrate the effectiveness of our method in reducing the semantic gap between features to be fused. As a result, our MASANet has achieved better performance on both public and our self-developed datasets, compared to other state-of-the-art infrared target segmentation methods. Codes and the dataset will be released at https://github.com/twentyfiveYang/MASANet_CITY-IRTD.
1 January 2022 in International Journal of Advanced Robotic Systems
Many robot manipulation processes involve large visual range variation between the hand-eye camera and the object, which in turn causes object scale change of a large span in the image sequence captured by the camera. In order to accurately guide the manipulator, the relative 6 degree of freedom (6D) pose between the object and manipulator is continuously required in the process. The large-span scale change of the object in the image sequence often leads to the 6D pose tracking failure of the object for existing pose tracking methods. To tackle this problem, this article proposes a novel scale-adaptive region-based monocular pose tracking method. Firstly, the impact of the object scale on the convergence performance of the local region-based pose tracker is meticulously tested and analyzed. Then, a universal region radius calculation model based on object scale is built based on the statical analysis result. Finally, we develop a novel scale-adaptive localized region-based pose tracking model by merging the scale-adaptive radius selection mechanism into the local region-based method. The proposed method adjusts local region size according to the scale of the object projection and achieves robust pose tracking. Experiment results on synthetic and real image sequences indicate that the proposed method achieves better performance over the traditional localized region-based method in manipulator operation scenarios which involve large visual range variation.