Three dimensional (3D) body scanning has been of great interest to many fields, yet it is still a challenge to generate accurate human body models in a convenient manner. In this paper, we present an accurate 3D body scanning system based on structured light technology. A four-step phase shifting combined with Gray-code method is applied to match pixels in camera and projector planes. The calculation of 3D point coordinates are also derived. The main contribution of this paper is twofold. First, an improved registration algorithm is proposed to align point clouds reconstructed from different views. Second, we propose a graph optimization algorithm to further minimize registration errors. Experimental results demonstrate that our system can produce accurate 3D body models conveniently.
31 July 2023 in IEEE Transactions on Intelligent Vehicles
Monocular depth estimation has attracted extensive attention and made great progress in recent years. However, the performance still lags far behind LiDAR-based depth completion algorithms. This is because the completion algorithms not only utilize the RGB image, but also have the prior of sparse depth collected by LiDAR. To reduce this performance gap, we propose a novel initiative that incorporates the concept of pseudo-LiDAR into depth estimation. The pseudo-LiDAR depends only on the camera and thus achieves a lower cost than LiDAR. To emulate the scan pattern of LiDAR, geometric sampling and appearance sampling are proposed. The former measures the vertical and horizontal azimuths of 3D scene points to establish the geometric correlation. The latter helps determine which “ pseudo-LiDAR rays” return an answer and which do not. Then, we build a sparse pseudo-LiDAR-based depth estimation framework. Extensive experiments show that the proposed method surpasses previous state-of-the-art competitors on the KITTI, NYU-Depth-v2 and SUN RGB-D datasets. The source code and trained models will be publicly available upon acceptance.
The closed-loop constraint provided by loop closure detection plays an essential role in eliminating accumulated errors of simultaneous localization and mapping systems. BEVLCD+: A real-time and rotation-invariant loop closure detection method based on bird’s eye view (BEV) of point cloud has been proposed in this paper. Projecting 3D point cloud to 2D BEV allows for a remarkable compression of data, resulting in improved processing efficiency using a convolutional neural network and facilitating the integration of point-level verification of loops through spatial information efficiently. Traditional convolution and pooling modules are improved to extract rotation-invariant features, ensuring that our BEVLCD+ method is rotation-invariant. Global descriptors aggregated by NetVLAD are used for fast retrieval, and local features are used for accurate verification. Local feature matching and RANSAC-based SVD provide the 3-degrees of freedom pose of loop pairs, which include translation and yaw in the horizontal direction. Experiments on benchmark datasets prove the excellent performance of our method.
1 October 2020 in Journal of Applied Remote Sensing
<!-- *** Custom HTML *** --> The rapid development of the aerospace industry has significantly increased the demand for remote-sensing images with high resolution and quality. Generating images with expected resolution from the samples obtained by common acquisition devices is a challenging task as the trade-off between cost and efficiency must be considered. We propose a super-resolution (SR) algorithm especially for remote-sensing images that is based on generative adversarial networks optimized by a classifier, which is called classifier-based super-resolution generative adversarial network (CSRGAN). We hypothesize that the confidence scores of classification can be a critical factor for representing the features in target remote-sensing images. To sufficiently take this factor into account during training, we add the class-score as an error into the loss function in addition to mean square error and high-dimensional features extracted from deep neural networks. Then, the classifier is utilized for both better SR performance and more precise classification. The classifier-testing branch of our system can also be flexibly combined with other network architectures to optimize SR performance on remote-sensing images. We validate the model on the NWPU-RESISC45 dataset considering both SR and classification performance. The final analysis is also provided and shows that the proposed CSRGAN outperforms existing algorithms.
6 February 2024 in IEEE Robotics and Automation Letters
Loop closure detection (LCD) plays a crucial role in simultaneous localization and mapping (SLAM) systems to eliminate accumulated odometry drifts as the map is built, and using multi-modal information can improve the accuracy and robustness of this system compared to single sensor. However, traditional fusion methods often require sophisticated space alignment of the sensors, which places high requirements on hardware equipment. Moreover, these methods fuse the point cloud and image in a weak manner, such as concentrating two kinds of features using calibration information, which makes they cannot take full use of the multi-modal information. In this letter, we propose a method for fusing asymmetric point clouds and images to detect loops. Bird's Eye View (BEV) of point cloud and image achieve information interaction and associations through learnable modules, rather than through hard intrinsic and extrinsic matrices to perform cross-fusion. Multi-information is fused in BEV grids, then the features of BEV key points are enhanced. Through experiments, it is found that under the training of this fusion strategy, the image information can be encoded into the BEV processing modules, and ultimately the performance of the method can be improved without images. The proposed method is evaluated on KITTI and KITTI-360, and the results demonstrate the state-of-the-art performance and remarkable efficiency.