Visual tracking is a vital component of autonomous driving perception system. Siamese networks have achieved great success in both accuracy and speed for visual tracking tasks. These Siamese trackers share a similar framework in which each tracker consists of two network branches for exploring semantic information. However, the performance of Siamese trackers is limited by an insufficient semantic template and an unsatisfactory updating strategy. To tackle these problems, we propose a manifold Siamese network for visual tracking that can simultaneously utilize semantic and geometric information. A manifold sample pool is constructed to exploit the manifold structure of image object sequences. This sample pool is dynamically learned via a fast Gaussian mixture model (GMM). After obtaining a manifold sample template, we design a deep architecture based on a correlation filter (CF) network and append a novel manifold feature branch. The network remains fully convolutional and can train a template to discriminate exemplar image and arbitrarily size search image. Then, a triplet occlusion score function cooperates with an effective update method that is established to prevent model drift. Extensive experiments show that the proposed tracking algorithm performs favorably compared with the state-of-the-art methods on three standard benchmark datasets at a high framerate, which is very suitable for autonomous driving.
16 January 2024 in IEEE Transactions on Intelligent Transportation Systems
Multi-modal fusion can take advantage of the LiDAR and camera to boost the robustness and performance of 3D object detection. However, there are still of great challenges to comprehensively exploit image information and perform accurate diverse feature interaction fusion. In this paper, we proposed a novel multi-modal framework, namely Point-Pixel Fusion for Multi-Modal 3D Object Detection (PPF-Det). The PPF-Det consists of three submodules, Multi Pixel Perception (MPP), Shared Combined Point Feature Encoder (SCPFE), and Point-Voxel-Wise Triple Attention Fusion (PVW-TAF) to address the above problems. Firstly, MPP can make full use of image semantic information to mitigate the problem of resolution mismatch between point cloud and image. In addition, we proposed SCPFE to preliminary extract point cloud features and point-pixel features simultaneously reducing time-consuming on 3D space. Lastly, we proposed a fine alignment fusion strategy PVW-TAF to generate multi-level voxel-fused features based on attention mechanism. Extensive experiments on KITTI benchmarks, conducted on September 24, 2023, demonstrate that our method shows excellent performance.
Accurate pedestrian trajectory prediction is a crucial task for ensuring the safety of autonomous driving. However, most of the existing methods only model pedestrian trajectories in the spatial-temporal domain, which results in a lack of analysis of motion at different scales. In this work, we propose a framework based on wavelet transform and diffusion model, which is called DiffWT. Different from previous approaches, our method employs a discrete wavelet transform (DWT) to perform time-frequency analysis of trajectories. The high-frequency component of the DWT indicates local motion details of the trajectory, while the low-frequency one represents overall motion trends. Secondly, we propose a trajectory decoder comprising a conditional diffusion model and a cross-constrained bi-directional trajectory generator (C2Bid). The diffusion model generates the distribution of implicit pedestrian behaviors by taking the multi-scale motion behaviors as conditions. Furthermore, the C2Bid module is designed as a cross-constrained bi-directional structure to decode behavioral distribution into multimodal trajectories. This trajectory decoder can generate precise distribution of trajectories and reduce accumulation of prediction errors. Extensive experimental results on the ETH/UCY and the Stanford drone datasets (SDD) demonstrate that our method achieves better performance as well as higher efficiency compared to other state-of-the-art approaches.
1 July 2024 in Journal of Colloid and Interface Science
Well-trained visual object detectors are generally confronted with a severe performance decline when deployed in a novel driving scenario due to the impact of domain shift. Despite excellent improvements in unsupervised domain adaptive object detection achieved by adversarial training, those approaches fail to capture the transfer core underlying the holistic scenes. To solve this problem, we propose a progressive critical region transfer framework for cross-domain visual object detection. Specifically, we exploit a potential foreground mining (PFM) module and a semantic-specific RoI aggregation (SRA) module to improve the robustness of the cross-domain detection framework. Upon the critical regions in the broad sense, the PFM module first highlights the foreground regions by reweighting the hierarchical feature maps in sequence, and then modifies location biases at the downstream position of the backbone network for more accurate upstream predictions. Deep into the critical regions in the narrow sense, the SRA module concentrates on establishing an appropriate matching between batch-wise RoIs and all semantic centers, and further strengthens the aggregation of cross-domain identical semantic with the complement of context references. Together these modules are obligated to transform the adaptation importance from the whole scope to the latent foreground areas, and afterward to the informative regions of interest along the detection pipeline. Experiments show that our progressive critical region transfer framework achieves a state-of-the-art performance in adverse weather, camera configuration, and complicated scene adaptation, which outperforms the baselines by 19.4%, 5.0%, and 6.1%, respectively.