Self-driving cars have experienced rapid development in the past few years, and Simultaneous Localization and Mapping (SLAM) is considered to be their basic capabilities. In this article, we propose a direct vision LiDAR fusion SLAM framework that consists of three modules. Firstly, a two-staged direct visual odometry module, which consists of a frame-to-frame tracking step, and an improved sliding window based thinning step, is proposed to estimate the accurate pose of the camera while maintaining efficiency. Secondly, every time a keyframe is generated, a dynamic objects considered LiDAR mapping module is utilized to refine the pose of the keyframe to obtain higher positioning accuracy and better robustness. Finally, a Parallel Global and Local Search Loop Closure Detection (PGLS-LCD) module that combines visual Bag of Words (BoW) and LiDAR-Iris feature is applied for place recognition to correct the accumulated drift and maintain a globally consistent map. We conducted a large number of experiments on the public dataset and our mobile robot dataset to verify the effectiveness of each module in our framework. Experimental results show that the proposed algorithm achieves more accurate pose estimation than the state-of-the-art methods.

Single-image super-resolution (SR) is an effective approach to enhance spatial resolution for numerous applications such as object detection and classification when the resolution of sensors is limited. Although deep convolutional neural networks (CNNs) proposed for this purpose in recent years have outperformed relatively shallow models, enormous parameters bring the risk of overfitting. In addition, due to the different scale of objects in images, the hierarchical features of deep CNN contain additional information for SR tasks, while most CNN models have not fully utilized these features. In this paper, we proposed a deep yet concise network to address these problems. Our network consists of two main structures: (1) recursive inference block based on dense connection reuse of local low-level features, and recursive learning is applied to control the model parameters while increasing the receptive fields; (2) a bidirectional convolutional LSTM (BiConvLSTM) layer is introduced to learn the correlations of features from each recursion and adaptively select the complementary information for the reconstruction layer. Experiments on multispectral satellite images, panchromatic satellite images, and nature high-resolution remote-sensing images showed that our proposed model outperformed state-of-the-art methods while utilizing fewer parameters, and ablation studies demonstrated the effectiveness of a BiConvLSTM layer for an image SR task.

Most SLAM (Simultaneous Localization and Mapping) algorithms are based on the assumption that the scene is static. However, in practice, most real scenes usually contain moving objects. In this letter, we introduce DymSLAM, a dynamic stereo visual SLAM system being capable of reconstructing a 4D (3D + time) dynamic scene with rigid moving objects. Unlike previous attempts that have considered moving objects as outliers and ignored them, DymSLAM obtains the 6DoF motion trajectory and 3D models about the dynamic objects. We segment motion models of different moving objects by a multi-motion segmentation approach and obtain the accurate masks of moving objects. Besides ego-motion, our system can obtain the 4D (3D + time) model and 6DoF trajectory of the moving object in the global reference frame while simultaneously reconstructing the dense map of the static background. Meanwhile, DymSLAM does not rely on semantic cues or prior knowledge and is suitable for unknown rigid objects. We conducted experiments in a real-world indoor environment where both the camera and the objects were moving in a wide range. The results proved that our proposed method is a state-of-the-art SLAM system for use in this dynamic environment.

Thermal infrared (TIR) remote-sensing imagery can allow objects to be imaged clearly at night through the long-wave infrared, so that the fusion of thermal infrared and visible (VIS) imagery is a way to improve the remote-sensing interpretation ability. However, due to the large radiation difference between the two kinds of images, it is very difficult to match them. One of the most important issues is the lack of comprehensive consideration of the modality-specific information and modality-shared information, which makes it difficult for the existing methods to obtain a modality-invariant feature representation. In this article, a cross-modality image matching network, which we refer to as CMM-Net, is proposed to realize thermal infrared and visible image matching by learning a modality-invariant feature representation. First, in order to extract the modality-specific features of the imagery, the framework constructs a shallow two-branch network to make full use of the modality-specific information, without sharing parameters. Second, in order to extract the high-level semantic information between the different modalities, modality-shared layers are embedded into the deep layers of the network. In addition, three novel loss functions are designed and combined to learn the modality-invariant feature representation, that is, the discriminative loss of the non-corresponding features in the same modality, the cross-modality loss of the corresponding features between different modalities, and the cross-modality triplet (CMT) loss. The multimodal matching experiments conducted with ground- and airborne-based thermal infrared images and visible images showed that the proposed method outperforms the existing image matching methods by about 2% and 6% for the ground and airborne images, respectively.

Different from object detection in natural image, optical remote sensing object detection is a challenging task, due to the diverse meteorological conditions, complex background, varied orientations, scale variations, etc. In this paper, to address this issue, we propose a novel object detection network (the global-local saliency constraint network, GLS-Net) that can make full use of the global semantic information and achieve more accurate oriented bounding boxes. More precisely, to improve the quality of the region proposals and bounding boxes, we first propose a saliency pyramid which combines a saliency algorithm with a feature pyramid network, to reduce the impact of complex background. Based on the saliency pyramid, we then propose a global attention module branch to enhance the semantic connection between the target and the global scenario. A fast feature fusion strategy is also used to combine the local object information based on the saliency pyramid with the global semantic information optimized by the attention mechanism. Finally, we use an angle-sensitive intersection over union (IoU) method to obtain a more accurate five-parameter representation of the oriented bounding boxes. Experiments with a publicly available object detection dataset for aerial images demonstrate that the proposed GLS-Net achieves a state-of-the-art detection performance.

Three-dimensional (3D) measurement of metal surfaces is one of fundamental tasks for product life-cycle management of complex equipment, which is meaningful but challenging due to its optical characteristics of highly reflectivity. To reliably reconstruct 3D metal surfaces, commonly-used techniques heavily rely on multiple exposures for optimal fusion but do not fit to high-efficient monitoring. To alleviate this reliance, we propose a novel single-exposure method called deep sinusoidal prior (DSP) for damaged phase recovery of highly reflective surfaces. Specifically, the sinusoidal hypothesis is instilled into an untrained deep neural network (DNN) as two-stream information, in order to bypass the problem of brightness enhancement. Utilizing elaborately designed loss functions, this approach enables us to restore the accurate phase encoding by fitting the DNN to two-stream sinusoidal priors. Experimental results demonstrate the proposed DSP method has superior performances on damaged phase recovery requiring no training samples. For instance, measuring a standard workpiece, absolute errors of the DSP method have been decreased substantially (81.69% and 59.49%) compared with the direct measurement and achieved similar accuracy (0.0754 mm vs. 0.0744 mm) compared with the reference. Most strikingly, the proposed method, for the first time, demonstrates a new perspective of recovering the reliable phase from a degraded one itself, contributing to the superior generalization capability insensitive to fringe frequencies, imaging settings and variant scenes.