In the computer vision field, many 3D deep learning models that directly manage 3D point clouds (proposed after PointNet) have been published. Moreover, deep learning-based-techniques have demonstrated state-of-the-art performance for supervised learning tasks on 3D point cloud data, such as classification and segmentation tasks for open datasets in competitions. Furthermore, many researchers have attempted to apply these deep learning-based techniques to 3D point clouds observed by aerial laser scanners (ALSs). However, most of these studies were developed for 3D point clouds without radiometric information. In this paper, we investigate the possibility of using a deep learning method to solve the semantic segmentation task of airborne full-waveform light detection and ranging (lidar) data that consists of geometric information and radiometric waveform data. Thus, we propose a data-driven semantic segmentation model called the full-waveform network (FWNet), which handles the waveform of full-waveform lidar data without any conversion process, such as projection onto a 2D grid or calculating handcrafted features. Our FWNet is based on a PointNet-based architecture, which can extract the local and global features of each input waveform data, along with its corresponding geographical coordinates. Subsequently, the classifier consists of 1D convolutional operational layers, which predict the class vector corresponding to the input waveform from the extracted local and global features. Our trained FWNet achieved higher scores in its recall, precision, and F1 score for unseen test data—higher scores than those of previously proposed methods in full-waveform lidar data analysis domain. Specifically, our FWNet achieved a mean recall of 0.73, a mean precision of 0.81, and a mean F1 score of 0.76. We further performed an ablation study, that is assessing the effectiveness of our proposed method, of the above-mentioned metric. Moreover, we investigated the effectiveness of our PointNet based local and global feature extraction method using the visualization of the feature vector. In this way, we have shown that our network for local and global feature extraction allows training with semantic segmentation without requiring expert knowledge on full-waveform lidar data or translation into 2D images or voxels.

Collapsed buildings should be detected with the highest priority during earthquake emergency response, due to the associated fatality rates. Although deep learning-based damage detection using vertical aerial images can achieve high performance, as depth information cannot be obtained, it is difficult to detect collapsed buildings when their roofs are not heavily damaged. Airborne LiDAR can efficiently obtain the 3D geometries of buildings (in the form of point clouds) and thus has greater potential to detect various collapsed buildings. However, there have been few previous studies on deep learning-based damage detection using point cloud data, due to a lack of large-scale datasets. Therefore, in this paper, we aim to develop a dataset tailored to point cloud-based building damage detection, in order to investigate the potential of point cloud data in collapsed building detection. Two types of building data are created: building roof and building patch, which contains the building and its surroundings. Comprehensive experiments are conducted under various data availability scenarios (pre–post-building patch, post-building roof, and post-building patch) with varying reference data. The pre–post scenario tries to detect damage using pre-event and post-event data, whereas post-building patch and roof only use post-event data. Damage detection is implemented using both basic and modern 3D point cloud-based deep learning algorithms. To adapt a single-input network, which can only accept one building’s data for a prediction, to the pre–post (double-input) scenario, a general extension framework is proposed. Moreover, a simple visual explanation method is proposed, in order to conduct sensitivity analyses for validating the reliability of model decisions under the post-only scenario. Finally, the generalization ability of the proposed approach is tested using buildings with different architectural styles acquired by a distinct sensor. The results show that point cloud-based methods can achieve high accuracy and are robust under training data reduction. The sensitivity analysis reveals that the trained models are able to locate roof deformations precisely, but have difficulty recognizing global damage, such as that relating to the roof inclination. Additionally, it is revealed that the model decisions are overly dependent on debris-like objects when surroundings information is available, which leads to misclassifications. By training on the developed dataset, the model can achieve moderate accuracy on another dataset with different architectural styles without additional training.

Most research on the extraction of earthquake-caused building damage using synthetic aperture radar (SAR) images used building damage certification assessments and the EMS-98-based evaluation as ground truth. However, these methods do not accurately assess the damage characteristics. The buildings identified as Major damage in the Japanese damage certification survey contain damage with various characteristics. If Major damage is treated as a single class, the parameters of SAR images will vary greatly, and the relationship between building damage and SAR images would not be properly evaluated. Therefore, it is necessary to divide Major damage buildings into more detailed classes. In this study, the Major damage buildings were newly classified into five damage classes, to correctly evaluate the relationship between building damage characteristics and SAR imagery. The proposed damage classification is based on Japanese damage assessment data and field photographs, and is classified according to the dominant damage characteristics of the building, such as collapse and damage to walls and roofs. We then analyzed the backscattering characteristics of SAR images for each classified damage class. We used ALOS-2 PALSAR-2 images observed before and after the 2016 Kumamoto earthquake in Mashiki Town, where many buildings were damaged by the earthquake. Then, we performed the analysis using two indices, the correlation coefficient R and the coherence differential value γ_dif, and the damage class. The results indicate that the backscattering characteristics of SAR images show different trends in each damage class. The R tended to decrease for large deformations such as collapsed buildings. The γ_dif was likely to be sensitive not only to collapsed buildings but also to damage with relatively small deformation, such as distortion and tilting. In addition, it was suggested that the ground displacement near the earthquake fault affected the coherence values.

Estimating surface normals from 3D point clouds is critical for various applications, including surface reconstruction and rendering. While existing methods for normal estimation perform well in regions where normals change slowly, they tend to fail where normals vary rapidly. To address this issue, we propose a novel approach called MSECNet, which improves estimation in normal varying regions by treating normal variation modeling as an edge detection problem. MSECNet consists of a backbone network and a multi-scale edge conditioning (MSEC) stream. The MSEC stream achieves robust edge detection through multi-scale feature fusion and adaptive edge detection. The detected edges are then combined with the output of the backbone network using the edge conditioning module to produce edge-aware representations. Extensive experiments show that MSECNet outperforms existing methods on both synthetic (PCPNet) and real-world (SceneNN) datasets while running significantly faster. We also conduct various analyses to investigate the contribution of each component in the MSEC stream. Finally, we demonstrate the effectiveness of our approach in surface reconstruction.

High-definition (HD) maps for autonomous driving rely on data from mobile mapping systems (MMS), but the high cost of MMS sensors has led researchers to explore cheaper alternatives like low-cost LiDAR sensors. While cost effective, these sensors produce sparser point clouds, leading to poor feature representation and degraded performance in deep learning techniques, such as convolutional neural networks (CNN), for tasks like road marking extraction and classification, which are essential for HD map generation. Examining common image segmentation workflows and the structure of U-Net, a CNN, reveals a source of performance loss in the succession of resizing operations, which further diminishes the already poorly represented features. Addressing this, we propose improving U-Net’s ability to extract and classify road markings from sparse-point-cloud-derived images by introducing a learnable resizer (LR) at the input stage and learnable resizer blocks (LRBs) throughout the network, thereby mitigating feature and localization degradation from resizing operations in the deep learning framework. Additionally, we incorporate Laplacian filters (LFs) to better manage activations along feature boundaries. Our analysis demonstrates significant improvements, with F1-scores increasing from below 20% to above 75%, showing the effectiveness of our approach in improving road marking extraction and classification from sparse-point-cloud-derived imagery.