3D Multi-object tracking (MOT) empowers mobile robots to accomplish well-informed motion planning and navigation tasks by providing motion trajectories of surrounding objects. However, existing 3D MOT methods typically employ a single similarity metric and physical model to perform data association and state estimation for all objects. With large-scale modern datasets and real scenes, there are a variety of object categories that commonly exhibit distinctive geometric properties and motion patterns. In this way, such distinctions would enable various object categories to behave differently under the same standard, resulting in erroneous matches between trajectories and detections, and jeopardizing the reliability of downstream tasks (navigation, etc.). Towards this end, we propose Poly-MOT, an efficient 3D MOT method based on the Tracking-By-Detection framework that enables the tracker to choose the most appropriate tracking criteria for each object category. Specifically, Poly-MOT leverages different motion models for various object categories to characterize distinct types of motion accurately. We also introduce the constraint of the rigid structure of objects into a specific motion model to accurately describe the highly nonlinear motion of the object. Additionally, we introduce a two-stage data association strategy to ensure that objects can find the optimal similarity metric from three custom metrics for their categories and reduce missing matches. On the NuScenes dataset, our proposed method achieves state-of-the-art performance with 75.4% AMOTA. The code is available at https://github.com/lixiaoyu20001P0Iy-MOT.

In this work, we introduce FARP-Net, an adaptive local-global feature aggregation and relation-aware proposal network for high-quality 3D object detection from pure point clouds. Our key insight is that learning adaptive local-global feature aggregation from an irregular yet sparse point cloud and generating superb proposals are both pivotal for detection. Technically, we propose a novel local-global feature aggregation layer (LGFAL) that fully exploits the complementary correlation between local features and global features, and fuses their strengths adaptively via an attention-based fusion module. Furthermore, we incorporate a lightweight feature affine module (LFAM) into LGFAL to map the local features into a normal distribution, thus acquiring fine-grained features of each local region in a weight-sharing manner. During object proposal generation, we propose a weighted relation-aware proposal module (WRPM) that uses an objectness-aware formalism to weigh the relation importance among object candidates for a clear and principal context, thereby facilitating the generation of high-quality proposals. The WRPM challenges the traditional practice of extracting contextual information among all object candidates, which is inefficient as object candidates are always noisy and redundant. Experimentally, FARP-Net delivers superior performance on two widely used benchmarks with fewer parameters, 64.0% mAP@0.25 on the SUN RGB-D dataset and 70.9% mAP@0.25 on the ScanNet V2 dataset. We further validate that the proposed LGFAL and WRPM can be integrated into both indoor and outdoor detectors to boost performance. The code is available at https://github.com/XT-1997/FARP-Net .

As scene coordinate regression (SCoRe) methods become prevailing in the area of visual camera localization, the issue of repetitive or sparse texture scenes continues to be a concern. Specifically, they will suffer from performance degeneration due to ambiguous patterns caused by visual similarity. In this work, we propose a novel network for camera localization through a single RGB image, with our key insight that taking only high-level feature maps as input can be difficult for the network to accurately model the regression problem due to ambiguous patterns and utilizing the rich spatial details in low-level feature maps can tackle this issue. The core components of the network are 1) Deep Feature Aggregation Module (DFAM), which eliminates the difference among the different level feature representations and fuses multi-level context information; 2) CoordConv Scheme, which further improves the discrimination of features in repetitive or sparse texture areas of the image; 3) Deep Supervision, which endows low-level feature maps with direct supervision from the ground truth to improve the accuracy of camera localization; 4) Uncertainty Modeling, which quantifies the prediction errors stemming from the intrinsic noise in the data. Moreover, to maximize the power of DFAM, we embed channel attention modules into it to prune redundant and noisy features, through which we can refine the different level feature maps. Our network is designed to be lightweight and efficient, and the proposed DFAM can be integrated into general SCoRe-based networks. Comprehensive experiments demonstrate the effectiveness of DFAM and the superiority of our network over the state-of-the-art methods on two benchmarks.

Recent advances in 3D object detection typically learn voxel-based or point-based representations on point clouds. Point-based methods preserve precise point positions but incur high computational load, whereas voxel-based methods rasterize unordered points into voxel grids efficiently but give rise to an accuracy bottleneck. To take advantage of voxel-and point-based representations, we develop an effective and efficient 3D object detector via a novel voxel-point geometry abstraction scheme. Our motivation is to use coarse voxel representation to accelerate proposal generation while using precise point representation to facilitate proposal refinement. For voxel representation learning, we propose a context enrichment module with a novel 3D sparse interpolation layer to augment raw points with multi-scale context. We further develop a point-based RoI pooling module with explicit position augmentation for proposal refinement. Extensive experiments on the widely used KITTI Dataset and the latest Waymo Open Dataset show that the proposed algorithm outperforms state-of-the-art point-voxel-based methods while running at 24 FPS on the TITAN XP GPU.