We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of nonsaliency. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the image amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale is equivalent to an image saliency detector. The saliency map is obtained by reconstructing the 2D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention.

Dense depth perception is critical for autonomous driving and other robotics applications. However, modern LiDAR sensors only provide sparse depth measurement. It is thus necessary to complete the sparse LiDAR data, where a synchronized guidance RGB image is often used to facilitate this completion. Many neural networks have been designed for this task. However, they often naïvely fuse the LiDAR data and RGB image information by performing feature concatenation or element-wise addition. Inspired by the guided image filtering, we design a novel guided network to predict kernel weights from the guidance image. These predicted kernels are then applied to extract the depth image features. In this way, our network generates content-dependent and spatially-variant kernels for multi-modal feature fusion. Dynamically generated spatially-variant kernels could lead to prohibitive GPU memory consumption and computation overhead. We further design a convolution factorization to reduce computation and memory consumption. The GPU memory reduction makes it possible for feature fusion to work in multi-stage scheme. We conduct comprehensive experiments to verify our method on real-world outdoor, indoor and synthetic datasets. Our method produces strong results. It outperforms state-of-the-art methods on the NYUv2 dataset and ranks 1st on the KITTI depth completion benchmark at the time of submission. It also presents strong generalization capability under different 3D point densities, various lighting and weather conditions as well as cross-dataset evaluations. The code will be released for reproduction.

Due to the difference of data modalities, it’s a very challenging task to find the feature correspondences between 2D and 3D data in LiDAR-Camera calibration. In existing works, the establishment of the cross-model correspondence is always simplified by specifically designing artificial targets or restricting the region of searching correspondences with the help of initial extrinsic parameters. To achieve automatic LiDAR-Camera calibration without prior knowledge, we propose a novel self-adaptive LiDAR-Camera calibration approach named ATOP which realizes a cascaded procedure of ATtention-to-OPtimization. In the attention stage, an attention-based object-level matching network called Cross-Modal Matching Network (CMON) is designed for finding the overlapped FOV(Field of View) between camera and LiDAR, and producing 2D-3D object-level correspondences. In the optimization stage, two cascaded PSO-based (Particle Swarm Optimization) algorithms, namely Point-PSO and Pose-PSO, are designed to estimate the LiDAR-Camera extrinsic parameters. Different from previous works, the proposed calibration method does not require any artificial targets or initial pose guesses, therefore it can be applied to achieve online self-adaptive LiDAR-Camera calibration. Besides, this is the first work, to our best knowledge, to achieve object-level matching between uncalibrated camera and LiDAR data. Experimental results on both the collected datasets and KITTI datasets demonstrate the effectiveness of the proposed method.

Recent advances in lane detection based on deep neural networks have enhanced the detection performance of intelligent vehicles under different traffic scenes. However, its still difficult for existing lane detection algorithms to robustly extract lane instances and adapt to varying lane numbers simultaneously. In this paper, we focus on anchor-free methods and propose an adaptive multi-lane detection approach based on instance segmentation for intelligent vehicles. By combining instance segmentation with lane center estimation, the proposed approach achieves robust and efficient performance. Besides, cosine-metric is incorporated into the objective functions that enables the proposed method to extract more discriminative features for foreground extraction. Different from the proposed approach, most existing algorithms treat the lane instance detection task as a multi-class object detection problem which require to predefine different lane categories and fix the lane number in advance, while the lane category predefinition degrades the robustness of the algorithms. Experimental results on public datasets demonstrate that the proposed approach achieves better overall performance compared with the state-of-the-art anchor free lane detection methods. It is also illustrated that the proposed approach can be easily extended to other instance segmentation tasks, e.g. vehicle segmentation. The code will be released at https://github.com/SYVAE/CenLaneNet.

Recent studies in remote sensing object detection have made excellent progress and shown promising performance. However, most current detectors only explore rotation-invariant feature extraction but disregard the valuable spatial and semantic prior knowledge in remote sensing images (RSIs), which limits the detection performance when encountering blurred or heavy occluded objects. To address this issue, we propose a mask-reconstruction relation learning (MRRL) framework to learn such prior knowledge among objects and a consistency-reasoning transformer over relation proposals (CTRP) to recognize objects with limited visual features via consistency reasoning. Specifically, MRRL framework applies random mask to some objects in the training dataset and performs masked objects reconstruction to guide the network to learn the distribution consistency of objects. CTRP is the core component of the MRRL framework, which models the interaction between spatial and semantic priors, and uses easy detected objects to reason hard detected objects. The trained CTRP can be integrated into the existing detector to improve the ability of object detection with limited visual features in RSIs. Extensive experiments on widely-used datasets for two distinct tasks, namely remote sensing object detection task and occluded object detection task, demonstrate the effectiveness of the proposed method. Source code is available at https://github.com/sunpeng96/CTRP_mmrotate.

In recent years, dominant multi-object tracking (MOT) and segmentation (MOTS) methods mainly follow the tracking-by-detection paradigm. Transformer-based end-to-end (E2E) solutions bring some ideas to MOT and MOTS, but they can not achieve a new state-of-the-art (SOTA) performance in major MOT and MOTS bench-marks. Detection and association are two main modules of the tracking-by-detection paradigm. Association techniques mainly depend on the combination of motion and appearance information. As deep learning has been recently developed, the performance of the detection and appearance model is rapidly improved. These trends made us consider whether we can achieve SOTA based on only high-performance detection and appearance model. Our paper mainly focuses on exploring this direction based on CBNetV2 with Swin-B as a detection model and MoCo-v2 as a self-supervised appearance model. Motion information and IoU mapping were removed during the association. Our method achieves SOTA results on 2 mainstream MOT datasets and 1 MOTS dataset which is BDD100K MOT, WAYMO 2D Tracking, BDD100K MOTS. Our method yielded a significant improvement of +10.7% and +33.7%, respectively on BDD 100K MOT and MOTS benchmark. The proposed method won first place in BDD100K Multiple Object Tracking (MOT) challenges at CVPR 2022 Workshop on Autonomous Driving. Our method also won first place in BDD100K Multiple Object Tracking (MOT) and Multiple Object Tracking and Segmentation (MOTS) challenges at ECCV 2022 Self-supervised Learning for Next-Generation Industry-level Autonomous Driving (SSLAD) Workshop. We hope our simple and effective method can give some insights to the MOT and MOTS research community. Source code will be released under this git repository https://github.com/CarlHuangNuc/MOT_MOTS_without_motion.