Semantic segmentation of large-scale point clouds is an indispensable component of outdoor scene perception, providing essential 3D semantic insights for applications in scene reconstruction, urban planning, autonomous driving, and more. However, the discriminative capability of point clouds features declines with increasing distance from the sensor, causing current methods to usually perform poorly in segmenting distant objects. To overcome this challenge and improve the differentiation between classes with similar geometric features, we propose the Cross-modal Semantic Focus Network (CSFNet). Firstly, we design a Multi-scale feature Dynamic Fusion module (MDF) to leverage multi-scale image features, thereby enriching the feature representation of point clouds with additional images color and texture information. Then, in order to extract the distinguishing features of distant and different categories of objects more efficiently, we propose a Semantic Focus Module (SFM) that employs a multi-class contrastive learning strategy to enhance feature discrimination. Finally, we introduce cross-modal knowledge distillation to augment the model’s comprehension of point clouds. Extensive experiments conducted on the SemanticKITTI and nuScenes datasets demonstrate the effectiveness of our method. Notably, our method achieves superior segmentation accuracy across multiple classes at various distances compared to current methods.

Recent advances in camera-equipped drone applications increased the demand for visual object detection algorithms with deep learning for aerial images. There are several limitations in accuracy for a single deep learning model. Inspired by ensemble learning can significantly improve the generalization ability of the model in the machine learning field, we introduce a novel integration strategy to combine the inference results of two different methods without non-maximum suppression. In this paper, a global and local ensemble network (GLE-Net) was proposed to increase the quality of predictions by considering the global weights for different models and adjusting the local weights for bounding boxes. Specifically, the global module assigns different weights to models. In the local module, we group the bounding boxes that corresponding to the same object as a cluster. Each cluster generates a final predict box and assigns the highest score in the cluster as the score of the final predict box. Experiments on benchmarks VisDrone2019 show promising performance of GLE-Net compared with the baseline network.

Stereo matching is a prominent research area in autonomous driving and computer vision. Despite significant progress made by learning-based methods, accurately predicting disparities in hazardous regions, which is crucial for ensuring safe vehicle operation, remains challenging. The limitations of methods based on Convolutional Neural Networks (CNNs) are most noticeable in textureless regions and repetitive patterns, leading to unreliable predictions. Furthermore, calculating disparities for boundaries and thin structures, where the disparity jump phenomenon is prominent remains difficult. To address these issues, we propose a lightweight stereo matching architecture that focuses on obtaining real-time and high-precision disparity maps in hazardous areas. We exploit an efficient global enhanced path to provide global representations in ill-posed regions, where CNN-based approaches often struggle. Second, our model integrates local and global features to generate more reliable cost volume. Finally, our innovative uncertainty-aware module refines disparity, making full use of high-frequency detailed information and uncertainty attention, effectively preserving complex structures. Comprehensive experimental studies on SceneFlow demonstrate our method outperforms state-of-the-art methods, achieving an End-Point Error (EPE) of 0.47 with only 3.60M parameters. The effectiveness of our method speed-accuracy trade-off is further confirmed by competitive results obtained from the KITTI 2012 and KITTI 2015 experiments. Code is available at: https://github.com/YJLCV/Guard-Net .

This study proposes a tightly coupled multi-sensor Simultaneous Localization and Mapping (SLAM) framework that integrates RGB-D and inertial measurements to achieve highly accurate 6 degree of freedom (6DOF) metric localization in a variety of environments. Through the consideration of geometric consistency, inertial measurement unit constraints, and visual re-projection errors, we present visual-inertial-depth odometry (called VIDO), an efficient state estimation back-end, to minimise the cascading losses of all factors. Existing visual-inertial odometers rely on visual feature-based constraints to eliminate the translational displacement and angular drift produced by Inertial Measurement Unit (IMU) noise. To mitigate these constraints, we introduce the iterative closest point error of adjacent frames and update the state vectors of observed frames through the minimisation of the estimation errors of all sensors. Moreover, the closed-loop module allows for further optimization of the global attitude map to correct the long-term drift. For experiments, we collect an RGBD-inertial data set for a comprehensive evaluation of VID-SLAM. The data set contains RGB-D image pairs, IMU measurements, and two types of ground truth data. The experimental results show that VID-SLAM achieves state-of-the-art positioning accuracy and outperforms mainstream vSLAM solutions, including ElasticFusion, ORB-SLAM2, and VINS-Mono.

Object detection is a challenging computer vision task with numerous real-world applications. In recent years, the concept of the object relationship model has become helpful for object detection and has been verified and realized in deep learning. Nonetheless, most approaches to modeling object relations are limited to using the anchor-based algorithms; they cannot be directly migrated to the anchor-free frameworks. The reason is that the anchor-free algorithms are used to eliminate the complex design of anchors and predict heatmaps to represent the locations of keypoints of different object categories, without considering the relationship between keypoints. Therefore, to better fuse the information between the heatmap channels, it is important to model the visual relationship between keypoints. In this paper, we present a knowledge-driven network (KDNet)—a new architecture that can aggregate and model keypoint relations to augment object features for detection. Specifically, it processes a set of keypoints simultaneously through interactions between their local and geometric features, thereby allowing the modeling of their relationship. Finally, the updated heatmaps were used to obtain the corners of the objects and determine their positions. The experimental results conducted on the RIDER dataset confirm the effectiveness of the proposed KDNet, which significantly outperformed other state-of-the-art object detection methods.