Although great progress has been made in generic object detection by advanced deep learning techniques, detecting small objects from images is still a difficult and challenging problem in the field of computer vision due to the limited size, less appearance, and geometry cues, and the lack of large-scale datasets of small targets. Improving the performance of small object detection has a wider significance in many real-world applications, such as self-driving cars, unmanned aerial vehicles, and robotics. In this article, the first-ever survey of recent studies in deep learning-based small object detection is presented. Our review begins with a brief introduction of the four pillars for small object detection, including multiscale representation, contextual information, super-resolution, and region-proposal . Then, the collection of state-of-the-art datasets for small object detection is listed. The performance of different methods on these datasets is reported later. Moreover, the state-of-the-art small object detection networks are investigated along with a special focus on the differences and modifications to improve the detection performance comparing to generic object detection architectures. Finally, several promising directions and tasks for future work in small object detection are provided. Researchers can track up-to-date studies on this webpage available at: https://github.com/tjtum-chenlab/SmallObjectDetectionList .
Predicting the future can significantly improve the safety of intelligent vehicles, which is a key component in autonomous driving. 3D point clouds can accurately model 3D information of surrounding environment and are crucial for intelligent vehicles to perceive the scene. Therefore, prediction of 3D point clouds has great significance for intelligent vehicles, which can be utilized for numerous further applications. However, due to point clouds are unordered and unstructured, point cloud prediction is challenging and has not been deeply explored in current literature. In this paper, we propose a novel motion-based neural network named MoNet. The key idea of the proposed MoNet is to integrate motion features between two consecutive point clouds into the prediction pipeline. The introduction of motion features enables the model to more accurately capture the variations of motion information across frames and thus make better predictions for future motion. In addition, content features are introduced to model the spatial content of individual point clouds. A recurrent neural network named MotionRNN is proposed to capture the temporal correlations of both features. Moreover, an attention-based motion align module is proposed to address the problem of missing motion features in the inference pipeline. Extensive experiments on two large-scale outdoor LiDAR point cloud datasets demonstrate the performance of the proposed MoNet. Moreover, we perform experiments on applications using the predicted point clouds and the results indicate the great application potential of the proposed method.
27 September 2021 in IEEE Transactions on Vehicular Technology
The localization system is one of the key components of autonomous vehicles. Widely-used GNSS (low-cost) alone can not meet the centimeter accuracy and is unstable in urban environment. Typically LiDAR-based localization system is very computationally demanding due to the usage of the dense point cloud map. In this paper, we propose a novel lightweight LiDAR-based localization system for autonomous vehicle in this paper. The proposed system only relies on lightweight poles and curbs landmark map, which is highly robust and efficient compared to other localization systems. Poles and curbs are selected as landmarks because of their commonality and stability. We novelly propose a Branch-and-Bound (BnB)-based global optimization method to tackle the data association problem of poles. Motion decoupling is adopted to decouple translation and rotation to improve the efficiency of the BnB-based algorithm. Besides, we propose a new local grid map-based representation for curbs to make better use of curb information. Cost functions for pole and curb are defined respectively and then fused for the subsequent non-linear optimization method to obtain the vehicle location. Experiments on KITTI dataset and our self-collected dataset demonstrate the efficiency and accuracy of our system.
17 December 2018 in IEEE Transactions on Image Processing
Simultaneously determining the relative pose and correspondence between a set of 3D points and its 2D projection is a fundamental problem in computer vision, and the problem becomes more difficult when the point sets are contaminated by noise and outliers. Traditionally, this problem is solved by local optimization methods, which usually start from an initial guess of the pose and alternately optimize the pose and the correspondence. In this paper, we formulate the problem as optimizing the pose of the 3D points in the SE(3) space to make its 2D projection best align with the 2D point set, which is measured by the cardinality of the inlier set on the 2D projection plane. We propose four geometric bounds for the position of the projection of a 3D point on the 2D projection plane and solve the 2D-3D point set registration problem by combining a global optimal rotation search and a grid search of translation. Compared with existing global optimization approaches, the proposed method utilizes a different problem formulation and more efficiently searches the translation space, which improves the registration speed. Experiments with synthetic and real data showed that the proposed approach significantly outperformed state-of-the-art local and global methods.
7 April 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Point cloud registration is a fundamental problem in 3D computer vision. Previous learning-based methods for LiDAR point cloud registration can be categorized into two schemes: dense-to-dense matching methods and sparse-to-sparse matching methods. However, for large-scale outdoor LiDAR point clouds, solving dense point correspondences is time-consuming, whereas sparse keypoint matching easily suffers from keypoint detection error. In this paper, we propose SDMNet, a novel Sparse-to-Dense Matching Network for large-scale outdoor LiDAR point cloud registration. Specifically, SDMNet performs registration in two sequential stages: sparse matching stage and local-dense matching stage. In the sparse matching stage, we sample a set of sparse points from the source point cloud and then match them to the dense target point cloud using a spatial consistency enhanced soft matching network and a robust outlier rejection module. Furthermore, a novel neighborhood matching module is developed to incorporate local neighborhood consensus, significantly improving performance. The local-dense matching stage is followed for fine-grained performance, where dense correspondences are efficiently obtained by performing point matching in local spatial neighborhoods of high-confidence sparse correspondences. Extensive experiments on three large-scale outdoor LiDAR point cloud datasets demonstrate that the proposed SDMNet achieves state-of-the-art performance with high efficiency.