Automatic building extraction and delineation from high-resolution satellite imagery is an important but very challenging task, due to the extremely large diversity of building appearances. Nowadays, it is possible to use multiple high-resolution remote sensing data sources, which allow the integration of different information in order to improve the extraction accuracy of building outlines. Many algorithms are built on spectral-based or appearance-based criteria, from single or fused data sources, to perform the building footprint extraction. But the features for these algorithms are usually manually extracted, which limits their accuracy. Recently developed fully convolutional networks (FCNs), which are similar to normal convolutional neural networks (CNN), but the last fully connected layer is replaced by another convolution layer with a large “receptive field,” quickly became the state-of-the-art method for image recognition tasks, as they bring the possibility to perform dense pixelwise classification of input images. Based on these advantages, i.e., the automatic extraction of relevant features, and dense classification of images, we propose an end-to-end FCN, which effectively combines the spectral and height information from different data sources and automatically generates a full resolution binary building mask. Our architecture (Fused-FCN4s) consists of three parallel networks merged at a late stage, which helps propagating fine detailed information from earlier layers to higher levels, in order to produce an output with more accurate building outlines. The inputs to the proposed Fused-FCN4s are three-band (RGB) , panchromatic (PAN), and normalized digital surface model (nDSM) images. Experimental results demonstrate that the fusion of several networks is able to achieve excellent results on complex data. Moreover, the developed model was successfully applied to different cities to show its generalization capacity.

A simple but robust approach for complex building description and extraction from high-resolution remotely sensed imagery based on graph-based shape representation is proposed. Classical approaches for building extraction usually involve a complex grouping process of low-level primitive features and are not robust in the presence of noise. To overcome these drawbacks, this approach presents an efficient and robust solution by integrating edges and regions. First, a region segmentation method is applied to obtain the approximate shape of the building. Second, Hough transformation is employed to derive the two perpendicular line sets corresponding to the building boundary. Third, a subset of the intersectional nodes of the two line sets is utilized to construct a building structural graph, based on the analysis of grey value difference between the two sides of each line segment. Finally, a graph search algorithm is performed to retrieve all the cycles in the structural graph. The cycle corresponding to the building boundary is identified as the final building outline on the basis of its area. Two experiments were carried out to evaluate and validate this approach and experimental results confirm its effectiveness and robustness.

Remote sensing (RS) image segmentation is an essential step in geographic object-based image analysis (GEOBIA) to ultimately derive “meaningful objects”. While many segmentation methods exist, most of them are not efficient for large data sets. Thus, the goal of this research is to develop an efficient parallel multi-scale segmentation method for RS imagery by combining graph theory and the fractal net evolution approach (FNEA). Specifically, a minimum spanning tree (MST) algorithm in graph theory is proposed to be combined with a minimum heterogeneity rule (MHR) algorithm that is used in FNEA. The MST algorithm is used for the initial segmentation while the MHR algorithm is used for object merging. An efficient implementation of the segmentation strategy is presented using data partition and the “reverse searching-forward processing” chain based on message passing interface (MPI) parallel technology. Segmentation results of the proposed method using images from multiple sensors (airborne, SPECIM AISA EAGLE II, WorldView-2, RADARSAT-2) and different selected landscapes (residential/industrial, residential/agriculture) covering four test sites indicated its efficiency in accuracy and speed. We conclude that the proposed method is applicable and efficient for the segmentation of a variety of RS imagery (airborne optical, satellite optical, SAR, high-spectral), while the accuracy is comparable with that of the FNEA method.

Automatic crowd detection in aerial images is certainly a useful source of information to prevent crowd disasters in large complex scenarios of mass events. A number of publications employ regression-based methods for crowd counting and crowd density estimation. However, these methods work only when a correct manual count is available to serve as a reference. Therefore, it is the objective of this paper to detect high-density crowds in aerial images, where counting– or regression–based approaches would fail. We compare two texture–classification methodologies on a dataset of aerial image patches which are grouped into ranges of different crowd density. These methodologies are: (1) a Bag–of–words (BoW) model with two alternative local features encoded as Improved Fisher Vectors and (2) features based on a Gabor filter bank. Our results show that a classifier using either BoW or Gabor features can detect crowded image regions with 97% classification accuracy. In our tests of four classes of different crowd-density ranges, BoW–based features have a 5%–12% better accuracy than Gabor.

As a probabilistic distance between two probability density functions, Kullback–Leibler divergence is widely used in many applications, such as image retrieval and change detection. Unfortunately, for some models, e.g., Gaussian Mixture Models (GMMs), Kullback–Leibler divergence is not analytically tractable. One has to resort to approximation methods. A number of methods have been proposed to address this issue. In this article, we compare seven methods, namely Monte Carlo method, matched bound approximation, product of Gaussians, variational method, unscented transformation, Gaussian approximation and min-Gaussian approximation, for approximating the Kullback–Leibler divergence between two Gaussian mixture models for satellite image retrieval. Two experiments using two public data sets have been performed. The comparison is carried out in terms of retrieval accuracy and computational time.

As the data acquisition capabilities of earth observation (EO) satellites have been improved significantly, a large amount of high-resolution images are downlinked continuously to ground stations. The data volume increases rapidly beyond the users’ capability to access the information content of the data. Thus, interactive systems that allow fast indexing of high-resolution images based on image content are urgently needed. In this paper, we present an interactive learning system for semantic annotation and content mining at patch level. It mainly comprises four components: primitive feature extraction including both spatial and temporal features, relevance feedback based on active learning, a human machine communication (HMC) interface and data visualisation. To overcome the shortage of training samples and to speed up the convergence, active learning is employed in this system. Two core components of active learning are the classifier training using already labelled image patches, and the sample selection strategy which selects the most informative samples for manual labelling. These two components work alternatively, significantly reducing the labelling effort and achieving fast indexing. In addition, our data visualisation is particularly designed for multi-temporal and multi-sensor image indexing, where efficient visualisation plays a critical role. The system is applicable to both optical and synthetic aperture radar images. It can index patches and it can also discover temporal patterns in satellite image time series. Three typical case studies are included to show its wide use in EO applications.