Ultra-wide band (UWB) localization system suffers from deteriorating performance in complex scenes, especially in non-line-of-sight (NLOS) conditions. In order to improve the accuracy and robustness of the localization system in NLOS environments, we propose an end-to-end deep neural network with both distance and received signal strength (RSS) measurements. On one hand, high-level spatial-temporal features can be learned through the proposed network from both RSS and distance data, which benefits the localization performance. On the other hand, the proposed network is robust to the variance in the number of available anchors, leading to high adaptability to different scenes. Specifically, three modules are designed in the deep network: 1) a module based on convolutional neural network (CNN) is presented to extract the local spatial features from the input data, and the structure of this module lends itself to varying-dimension input. 2) To manage the correlations between consecutive frames, we develop a deep long short-term memory (LSTM) model to extract temporal features and provide a high-level representation for a series of input data. 3) Finally, the fully-connected layers are utilized to estimate the 3D positions of the UWB tag. We conduct extensive experiments in three real-world scenarios to evaluate the proposed deep network. The experimental results indicate that the proposed network can significantly improve the accuracy and robustness of the UWB localization results, especially in NLOS situations.

In this article, a resilient tightly coupled ultra-wideband (UWB) visual–inertial indoor localization system (R-UVIS) is developed to obtain accurate and robust localization performance in complex scenes, even in the case when sensors fail. More specifically, three schemes are designed for the proposed system. First, we introduce the line and image patch features to improve the precision and robustness of the visual features. Besides, we propose accurate loop closure and relocalization methods based on multifeatures to improve the performance of the localization system. Second, we introduce the UWB sensor into the system to suppress the localization drifts in the complex scenes and provide a fixed reference frame. Third, we propose a resilient multisensor fusion method based on an optimization framework to fuse the UWB, visual and inertial measurements in a tightly coupled manner. This data fusion approach improves the robustness of the system, making the localization system seamlessly switch among different localization modes depending on the specific scenes. In addition, an initialization process along with three sensors is also designed for the whole system. We conduct extensive experiments on the public dataset and in real-world scenarios to evaluate the proposed R-UVIS. The experimental results show that the proposed R-UVIS can provide the accurate and robust localization results in a fixed coordinate system for the complicated indoor scenes, even in the case when visual tracking fails or the UWB anchors are unavailable.

Deep convolutional neural networks (CNNs) have demonstrated tremendous success in image super-resolution (SR). According to the frequency principle, the vanilla CNNs fit the target function from low to high frequencies during the training process. It implies an implicit bias that CNNs tend to fit the training data by a low-frequency function. This is detrimental to SR task which essentially recovers the missing high-frequency cues from degraded images. To address this issue, a novel spatial-frequency domain hybrid network (SFHN) is proposed for image SR in this paper. More specifically, it contains multiple spatial-frequency domain hybrid convolution blocks (SFBlocks) for extracting both spatial and frequency components. In particular, frequency information is complementary to spatial one, mitigating the disadvantage of the spatial CNNs in capturing high-frequency content resulting from the inherent bias of the networks. In contrast to convolution blocks in the previous SR methods, the proposed SFBlock is configured with additional frequency-domain convolution branch driven by the Fourier transform, which allows to process frequency content. In addition, we introduce a spectral loss based on Fast Fourier Transform (FFT) to fully leverage the performance of our SFHN, as it prevents the loss of important frequency content during training. Experimental results on public benchmarks demonstrate that our SFHN achieves promising performance superior to the state-of-the-art methods.

With the three-dimensional (3D) coordinates of objects captured by a sequence of images taken in different views, object reconstruction is a technique which aims to recover the shape and appearance information of objects. Although great progress in object reconstruction has been made over the past few years, object reconstruction in occlusion situations remains a challenging problem. In this paper, we propose a novel method to reconstruct occluded objects based on synthetic aperture imaging. Unlike most existing methods, which either assume that there is no occlusion in the scene or remove the occlusion from the reconstructed result, our method uses the characteristics of synthetic aperture imaging that can effectively reduce the influence of occlusion to reconstruct the scene with occlusion. The proposed method labels occlusion pixels according to variance and reconstructs the 3D point cloud based on synthetic aperture imaging. Accuracies of the point cloud are tested by calculating the spatial difference between occlusion and non-occlusion conditions. The experiment results show that the proposed method can handle the occluded situation well and demonstrates a promising performance.