State Key Laboratory of Remote Sensing Science, Jointly Sponsored by Beijing Normal University and Institute of Remote Sensing and Digital Earth of Chinese Academy of Sciences, Beijing 100875, China
Mr. Chao Ji
State Key Laboratory of Remote Sensing Science, Jointly Sponsored by Beijing Normal University and Institute of Remote Sensing and Digital Earth of Chinese Academy of Sciences, Beijing 100875, China
Satellite mapping of buildings and built-up areas used to be delineated from high spatial resolution (e.g., meters or sub-meters) and middle spatial resolution (e.g., tens of meters or hundreds of meters) satellite images, respectively. To the best of our knowledge, it is important to explore a deep-learning approach to delineate high-resolution semantic maps of buildings from middle-resolution satellite images. The approach is termed as super-resolution semantic segmentation in this paper. Specifically, we design a neural network with integrated low-level image features of super-resolution and high-level semantic features of super-resolution, which is trained with Sentinel-2A images (i.e., 10 m) and higher-resolution semantic maps (i.e., 2.5 m). The network, based on super-resolution semantic segmentation features is called FSRSS-Net. In China, the 35 cities are partitioned into three groups, i.e., 19 cities for model training, four cities for quantitative testing and the other 12 cities for qualitative generalization ability analysis of the learned networks. A large-scale sample dataset is created and utilized to train and validate the performance of the FSRSS-Net, which includes 8597 training samples and 766 quantitative accuracy evaluation samples. Quantitative evaluation results show that: (1) based on the 10 m Sentinel-2A image, the FSRSS-Net can achieve super-resolution semantic segmentation and produce 2.5 m building recognition results, and there is little difference between the accuracy of 2.5 m results by FSRSS-Net and 10 m results by U-Net. More importantly, the 2.5 m building recognition results by FSRSS-Net have higher accuracy than the 2.5 m results by U-Net 10 m building recognition results interpolation up-sampling; (2) from the spatial visualization of the results, the building recognition results of 2.5 m are more precise than those of 10 m, and the outline of the building is better depicted. Qualitative analysis shows that: (1) the learned FSRSS-Net can be also well generalized to other cities that are far from training regions; (2) the FSRSS-Net can still achieve comparable results to the U-Net 2 m building recognition results, even when the U-Net is directly trained using both 2-meter resolution GF2 satellite images and corresponding semantic labels.
Stereo photogrammetric survey used to be used to extract the height of buildings, then to convert the height to number of stories through certain rules to estimate the number of stories of buildings by means of satellite remote sensing. In contrast, we propose a new method using deep learning to estimate the number of stories of buildings from monocular optical satellite image end to end in this paper. To the best of our knowledge, this is the first attempt to directly estimate the number of stories of buildings from monocular satellite images. Specifically, in the proposed method, we extend a classic object detection network, i.e., Mask R-CNN, by adding a new head to predict the number of stories of detected buildings from satellite images. GF-2 images from nine cities in China are used to validate the effectiveness of the proposed methods. The result of experiment show that the mean absolute error of prediction on buildings whose stories between 1–7, 8–20, and above 20 are 1.329, 3.546, and 8.317, respectively, which indicate that our method has possible application potentials in low-rise buildings, but the accuracy in middle-rise and high-rise buildings needs to be further improved.
The building damage caused by natural disasters seriously threatens human security. Applying deep learning algorithms to identify collapsed buildings from remote sensing images is crucial for rapid post-disaster emergency response. However, the diversity of buildings, limited training dataset size, and lack of ground-truth samples after sudden disasters can significantly reduce the generalization of a pre-trained model for building damage identification when applied directly to non-preset locations. To address this challenge, a self-incremental learning framework (i.e., SELF) is proposed in this paper, which can quickly improve the generalization ability of the pre-trained model in disaster areas by self-training an incremental model using automatically selected samples from post-disaster images. The effectiveness of the proposed method is verified on the 2010 Yushu earthquake, 2023 Turkey earthquake, and other disaster types. The experimental results demonstrate that our approach outperforms state-of-the-art methods in terms of collapsed building identification, with an average increase of more than 6.4% in the Kappa coefficient. Furthermore, the entire process of the self-incremental learning method, including sample selection, incremental learning, and collapsed building identification, can be completed within 6 h after obtaining the post-disaster images. Therefore, the proposed method is effective for emergency response to natural disasters, which can quickly improve the application effect of the deep learning model to provide more accurate building damage results.
Gross floor area is defined as the product of number of building stories and its base area. Gross floor area acquisition is the core problem to estimate floor area ratio, which is an important indicator for many geographical analyses. High data acquisition cost or inherent defect of methods for existing gross floor area acquisition methods limit their applications in a wide range. In this paper we proposed three instance-wise gross floor area estimation methods in various degrees of end-to-end learning from monocular optical images based on the NoS R-CNN, which is a deep convolutional neural network to estimate the number of building stories. To the best of our knowledge, this is the first attempt to estimate instance-wise gross floor area from monocular optical satellite images. For comparing the performance of the proposed three methods, experiments on our dataset from nine cities in China were carried out, and the results were analyzed in detail in order to explore the reasons for the performance gap between the different methods. The results show that there is an inverse relationship between the model performance and the degree of end-to-end learning for base area estimation task and gross floor area estimation task. The quantitative and qualitative evaluations of the proposed methods indicate that the performances of proposed methods for accurate GFA estimation are promising for potential applications using large-scale remote sensing images. The proposed methods provide a new perspective for gross floor area/floor area ratio estimation and downstream tasks such as population estimation, living conditions assessment, etc.