11 March 2024 in IEEE Transactions on Vehicular Technology
In the realm of multimodal multi-object tracking (MOT) applications based on point clouds and images, the current research predominantly focuses on enhancing tracking accuracy, often neglecting the issue of computational efficiency. Consequently, these models often struggle to exhibit optimal tracking capabilities in scenarios demanding high real-time performance. To address these challenges, this paper introduces a fast multi-object tracking model based on multimodal fusion (MF-Net). The model is divided into three primary modules: object detection, multimodal fusion, and trajectory matching. Firstly, a 2D detector is used to identify objects in the image and compute their posterior estimate, and a 3D classification network extracts the foreground points of the object from the point cloud. Subsequently, a perspective projection module is then designed to determine the transformation matrix and the minimum number of vertex pairs that map the coordinates of the foreground points onto a 2D plane. Based on the model, a Planar Gaussian Function (PGF) model was constructed to fit small and hard objects that were missed in the image according to the foreground points, thus compensating for the limitations of 2D detectors and ensuring accuracy while reducing training time. Finally, the merged object performs trajectory matching. The performance of MF-Net has been verified through experiments in plenty conducted on publicly available KITTI and nuScenes datasets. In comparison to existing competitive models, our algorithm demonstrates a substantial enhancement in both detection and tracking performance, achieving satisfactory accuracy but showcasing superior real-time efficiency. The MF-Net's source code is obtained at https://github.com/ShiruiTian/MFNet .
14 October 2025 in IEEE Transactions on Vehicular Technology
The object point clouds acquired by the original LiDAR are inherently sparse and incomplete, resulting in suboptimal single object tracking (SOT) precision for 3D bounding boxes, especially for small objects at extended ranges. Furthermore, the feature extraction process for object point clouds necessitates substantial computational resources, and the constraints of real-world scenarios often impede the transmission speed essential for practical deployment. Therefore, we propose a novel object-tracking network (THT-Net) based on global and local transformer hashing and tensor analysis, which comprises four key modules. Firstly, point cloud features are extracted using Siamese networks to compute the correlation between the object template region and the search region. Secondly, tensor decomposition and completion are employed to reduce the dimensionality of object point cloud features while preserving and extracting object information. Thirdly, a hash function is constructed to encode the positional information of point clouds, and global and local transformer modules are utilized to analyze the inter-object correlations and predict missing or distant complex objects. Finally, the prediction head module is implemented to reassemble the feature tensor and localize the object-bounding box, thereby finalizing the tracking process. We conduct numerous experiments on the KITTI and nuScenes datasets to evaluate the performance of THT-Net. The experimental results demonstrate that: 1) The THT-Net achieves average success rates and accuracy of 65.5% and 84.1% (KITTI), and 51.36% and 59.88% (nuScenes), respectively; 2) The THT-Net significantly reduces the algorithm training time to 2.06 hours (KITTI) and 3.85 hours (nuScenes). The source code is accessible at https://github.com/ShiruiTian/THTNet.
1 October 2026 in IEEE Transactions on Intelligent Transportation Systems