In this letter, we propose a pseudo-siamese convolutional neural network architecture that enables to solve the task of identifying corresponding patches in very high-resolution optical and synthetic aperture radar (SAR) remote sensing imagery. Using eight convolutional layers each in two parallel network streams, a fully connected layer for the fusion of the features learned in each stream, and a loss function based on binary cross entropy, we achieve a one-hot indication if two patches correspond or not. The network is trained and tested on an automatically generated data set that is based on a deterministic alignment of SAR and optical imagery via previously reconstructed and subsequently coregistered 3-D point clouds. The satellite images, from which the patches comprising our data set are extracted, show a complex urban scene containing many elevated objects (i.e., buildings), thus providing one of the most difficult experimental environments. The achieved results show that the network is able to predict corresponding patches with high accuracy, thus indicating great potential for further development toward a generalized multisensor key-point matching procedure.

• Combined big and multisensory Earth observation data with deep learning. • Revealed the first time detailed morphology of urban agglomerations across the globe. • Statistical study of our results quantifies a global inequality in population density. • Clustering of the global result identified seven unique urban morphological patterns. • This global urban morphological dataset (So2Sat Global Urban LCZ) will be open access.

This paper addresses the highly challenging problem of automatically detecting man-made structures especially buildings in very high-resolution (VHR) synthetic aperture radar (SAR) images. In this context, this paper has two major contributions. First, it presents a novel and generic workflow that initially classifies the spaceborne SAR tomography (TomoSAR) point clouds--generated by processing VHR SAR image stacks using advanced interferometric techniques known as TomoSAR--into buildings and nonbuildings with the aid of auxiliary information (i.e., either using openly available 2-D building footprints or adopting an optical image classification scheme) and later back project the extracted building points onto the SAR imaging coordinates to produce automatic large-scale benchmark labeled (buildings/nonbuildings) SAR data sets. Second, these labeled data sets (i.e., building masks) have been utilized to construct and train the state-of-the-art deep fully convolution neural networks with an additional conditional random field represented as a recurrent neural network to detect building regions in a single VHR SAR image. Such a cascaded formation has been successfully employed in computer vision and remote sensing fields for optical image classification but, to our knowledge, has not been applied to SAR images. The results of the building detection are illustrated and validated over a TerraSAR-X VHR spotlight SAR image covering approximately 39 km²--almost the whole city of Berlin-- with the mean pixel accuracies of around 93.84%.

This article addresses the question of mapping building functions jointly using both aerial and street view images via deep learning techniques. One of the central challenges here is determining a data fusion strategy that can cope with heterogeneous image modalities. We demonstrate that geometric combinations of the features of such two types of images, especially in an early stage of the convolutional layers, often lead to a destructive effect due to the spatial misalignment of the features. Therefore, we address this problem through a decision-level fusion of a diverse ensemble of models trained from each image type independently. In this way, the significant differences in appearance of aerial and street view images are taken into account. Compared to the common multi-stream end-to-end fusion approaches proposed in the literature, we are able to increase the precision scores from 68% to 76%. Another challenge is that sophisticated classification schemes needed for real applications are highly overlapping and not very well defined without sharp boundaries. As a consequence, classification using machine learning becomes significantly harder. In this work, we choose a highly compact classification scheme with four classes, commercial, residential, public, and industrial because such a classification has a very high value to urban geography being correlated with socio-demographic parameters such as population density and income.

Since its launch in 2007, TerraSAR-X has continuously provided spaceborne synthetic aperture radar (SAR) images of our planet with unprecedented spatial resolution, geodetic, and geometric accuracy. This has brought life to the once inscrutable SAR images, which deterred many researchers. Thanks to merits like higher spatial resolution and more precise orbit control, we are now able to indicate individual buildings, even individual floors, to pinpoint targets within centimeter accuracy. As a result, multi-baseline SAR interferometric (InSAR) techniques are flourishing, from point target-based algorithms, to coherent stacking techniques, to absolute positioning of the former techniques. This article reviews the recent advances of multi-baseline InSAR techniques using TerraSAR-X images. Particular focus was put on our own development of persistent scatterer interferometry, SAR tomography, robust estimation in distributed scatterer interferometry and absolute positioning using geodetic InSAR. Furthermore, by introducing the applications associated with these techniques, such as 3D reconstruction and deformation monitoring, this article is also intended to give guidance to wider audiences who would like to resort to SAR data and related techniques for their applications.