Recently, convolutional neural networks (CNN) have been intensively investigated for the classification of remote sensing data by extracting invariant and abstract features suitable for classification. In this paper, a novel framework is proposed for the fusion of hyperspectral images and LiDAR-derived elevation data based on CNN and composite kernels. First, extinction profiles are applied to both data sources in order to extract spatial and elevation features from hyperspectral and LiDAR-derived data, respectively. Second, a three-stream CNN is designed to extract informative spectral, spatial, and elevation features individually from both available sources. The combination of extinction profiles and CNN features enables us to jointly benefit from low-level and high-level features to improve classification performance. To fuse the heterogeneous spectral, spatial, and elevation features extracted by CNN, instead of a simple stacking strategy, a multi-sensor composite kernels (MCK) scheme is designed. This scheme helps us to achieve higher spectral, spatial, and elevation separability of the extracted features and effectively perform multi-sensor data fusion in kernel space. In this context, a support vector machine and extreme learning machine with their composite kernels version are employed to produce the final classification result. The proposed framework is carried out on two widely used data sets with different characteristics: an urban data set captured over Houston, USA, and a rural data set captured over Trento, Italy. The proposed framework yields the highest OA of and for Houston and Trento data sets. Experimental results confirm that the proposed fusion framework can produce competitive results in both urban and rural areas in terms of classification accuracy, and significantly mitigate the salt and pepper noise in classification maps.
Geographic Object-Based Image Analysis (GEOBIA) techniques have become increasingly popular in remote sensing. GEOBIA has been claimed to represent a paradigm shift in remote sensing interpretation. Still, GEOBIA—similar to other emerging paradigms—lacks formal expressions and objective modelling structures and in particular semantic classification methods using ontologies. This study has put forward an object-based semantic classification method for high resolution satellite imagery using an ontology that aims to fully exploit the advantages of ontology to GEOBIA. A three-step workflow has been introduced: ontology modelling, initial classification based on a data-driven machine learning method, and semantic classification based on knowledge-driven semantic rules. The classification part is based on data-driven machine learning, segmentation, feature selection, sample collection and an initial classification. Then, image objects are re-classified based on the ontological model whereby the semantic relations are expressed in the formal languages OWL and SWRL. The results show that the method with ontology—as compared to the decision tree classification without using the ontology—yielded minor statistical improvements in terms of accuracy for this particular image. However, this framework enhances existing GEOBIA methodologies: ontologies express and organize the whole structure of GEOBIA and allow establishing relations, particularly spatially explicit relations between objects as well as multi-scale/hierarchical relations.
19 September 2022 in PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science
Geospatial information science (GI science) is concerned with the development and application of geodetic and information science methods for modeling, acquiring, sharing, managing, exploring, analyzing, synthesizing, visualizing, and evaluating data on spatio-temporal phenomena related to the Earth. As an interdisciplinary scientific discipline, it focuses on developing and adapting information technologies to understand processes on the Earth and human-place interactions, to detect and predict trends and patterns in the observed data, and to support decision making. The authors – members of DGK, the Geoinformatics division, as part of the Committee on Geodesy of the Bavarian Academy of Sciences and Humanities, representing geodetic research and university teaching in Germany – have prepared this paper as a means to point out future research questions and directions in geospatial information science. For the different facets of geospatial information science, the state of art is presented and underlined with mostly own case studies. The paper thus illustrates which contributions the German GI community makes and which research perspectives arise in geospatial information science. The paper further demonstrates that GI science, with its expertise in data acquisition and interpretation, information modeling and management, integration, decision support, visualization, and dissemination, can help solve many of the grand challenges facing society today and in the future.
Remote sensing (RS) image segmentation is an essential step in geographic object-based image analysis (GEOBIA) to ultimately derive “meaningful objects”. While many segmentation methods exist, most of them are not efficient for large data sets. Thus, the goal of this research is to develop an efficient parallel multi-scale segmentation method for RS imagery by combining graph theory and the fractal net evolution approach (FNEA). Specifically, a minimum spanning tree (MST) algorithm in graph theory is proposed to be combined with a minimum heterogeneity rule (MHR) algorithm that is used in FNEA. The MST algorithm is used for the initial segmentation while the MHR algorithm is used for object merging. An efficient implementation of the segmentation strategy is presented using data partition and the “reverse searching-forward processing” chain based on message passing interface (MPI) parallel technology. Segmentation results of the proposed method using images from multiple sensors (airborne, SPECIM AISA EAGLE II, WorldView-2, RADARSAT-2) and different selected landscapes (residential/industrial, residential/agriculture) covering four test sites indicated its efficiency in accuracy and speed. We conclude that the proposed method is applicable and efficient for the segmentation of a variety of RS imagery (airborne optical, satellite optical, SAR, high-spectral), while the accuracy is comparable with that of the FNEA method.
24 May 2018 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Convolutional neural networks, or CNNs, raised the bar for most computer vision problems and have an increasing impact in remote sensing. However, since they usually contain multiple pooling layers, detection of exact borders of small objects at their original resolution remains yet a challenging topic. Additionally, efforts are being made to reduce the amount of training data. In this paper, we investigate the potential of fully convolutional neural networks (FCNs) for individual vehicle detection in combined elevation and optical data using relatively few training samples. By the proposed multibranch CNN, we combine object recognition within a deep learning framework with the object segmentation at a high resolution, for which two CNN branches are employed. Data fusion is accomplished with a pseudo-Siamese approach. The pixelwise classification likelihood, also referred to as heatmap, is harmoniously postprocessed by a vectorization module, which is based on the minimum bounding rectangle (MBR) extraction and allows for delineation of groups of vehicles. Two methods were developed in which MBRs are supported either by pairs of parallel lines or by region growing. Our approach allows efficient training with few training samples, while delivering high-quality detection results and good computational performance. In our detailed evaluation, we investigate the benefits of data fusion and compare our approach to other state-of-the-art networks. Different datasets were used, containing optical images and elevation data, derived either from airborne laser scanning or from photogrammetric reconstruction. The obtained results are very promising with F$_1$ scores up to 97%.
Since trees are a vital part of urban green infrastructure, automatic mapping of individual urban trees is becoming increasingly important for city management and planning. Although deep-learning-based object detection networks are the state-of-the-art in computer vision, their adaptation to individual tree detection in urban areas has scarcely been studied. Some existing works have employed 2D object detection networks for this purpose. However, these have used three-dimensional information only in the form of projected feature maps. In contrast, we exploited the full 3D potential of airborne laser scanning (ALS) point clouds by using a 3D neural network for individual tree detection. Specifically, a sparse convolutional network was used for 3D feature extraction, feeding both semantic segmentation and circular object detection outputs, which were combined for further increased accuracy. We demonstrate the capability of our approach on an urban topographic ALS point cloud with 10,864 hand-labeled ground truth trees. Our method achieved an average precision of 83% regarding the common 0.5 intersection over union criterion. 85% percent of the stems were found correctly with a precision of 88%, while tree area was covered by the individual tree detections with an F1 accuracy of 92%. Thereby, we outperformed traditional delineation baselines and recent detection networks.