2 October 2023 in IEEE Transactions on Geoscience and Remote Sensing
LiDAR and camera, as two different sensors, supply geometric (point clouds) and semantic (RGB images) information of 3D scenes. However, it is still challenging for existing methods to fuse data from the two cross sensors, making them complementary for quality 3D object detection (3OD). We propose ImLiDAR, a new 3OD paradigm to narrow the cross-sensor discrepancies by progressively fusing the multi-scale features of camera Images and LiDAR point clouds. ImLiDAR enables to provide the detection head with cross-sensor yet robustly fused features. To achieve this, two core designs exist in ImLiDAR. First, we propose a cross-sensor dynamic message propagation module to combine the best of the multi-scale image and point features. Second, we raise a direct set prediction problem that allows designing an effective set-based detector to tackle the inconsistency of the classification and localization confidences, and the sensitivity of hand-tuned hyperparameters. Besides, the novel set-based detector can be detachable and easily integrated into various detection networks. Comparisons on the KITTI, nuScenes, and SUN-RGBD datasets all show clear visual and numerical improvements of our ImLiDAR over forty-five state-of-the-art 3OD methods.
1 January 2023 in International Journal of Intelligent Systems
Object detection remains challenging in hazy weather conditions due to the poor visibility of captured images. There are currently two types of detectors capable of adapting to varying weather conditions: (i) low‐level adaptation methods that combine one detector with an additional dehazing network and (ii) high‐level adaptation methods that explore various kinds of domain adaptation knowledge. However, neither of these approaches can achieve desirable performance due to their inherent limitations. We raise an intriguing question—if combining both low‐level adaptation and high‐level adaptation, can improve the generalization ability of a detector in hazy weather conditions? To answer it, we propose a Joint High‐Low Adaptation Object Detection paradigm (HLA‐HOD) in hazy weather conditions. By combining both low‐level adaptation and high‐level adaptation, HLA‐HOD achieves superior performance on hazy images without requiring ground‐truth bounding boxes or clean images. Extensive experiments demonstrate that our method outperforms state‐of‐the‐art low‐level and high‐level adaptation methods by a large margin both quantitatively and qualitatively.
1 January 2024 in International Journal of Intelligent Systems
We propose a novel model‐free unsupervised learning paradigm to tackle the unfavorable prevailing problem of real‐world image deraining, dubbed MUL‐Derain. Beyond existing unsupervised deraining efforts, MUL‐Derain leverages a model‐free Multiscale Attentive Filtering (MSAF) to handle multiscale rain streaks. Therefore, formulation of any rain imaging is not necessary, and it requires neither iterative optimization nor progressive refinement operations. Meanwhile, MUL‐Derain can efficiently compute spatial coherence and global interactions by modeling long‐range dependencies, allowing MSAF to learn useful knowledge from a larger or even global rain region. Furthermore, we formulate a novel multiloss function to constrain MUL‐Derain to preserve both color and structure information from the rainy images. Extensive experiments on both synthetic and real‐world datasets demonstrate that our MUL‐Derain obtains state‐of‐the‐art performance over un/semisupervised methods and exhibits competitive advantages over the fully‐supervised ones.