Deep convolutional neural networks (DCNNs) have recently emerged as a dominant paradigm for machine learning in a variety of domains. However, acquiring a suitably large data set for training DCNN is often a significant challenge. This is a major issue in the remote sensing domain, where we have extremely large collections of satellite and aerial imagery, but lack the rich label information that is often readily available for other image modalities. In this letter, we investigate the use of DCNN for land-cover classification in high-resolution remote sensing imagery. To overcome the lack of massive labeled remote-sensing image data sets, we employ two techniques in conjunction with DCNN: transfer learning (TL) with fine-tuning and data augmentation tailored specifically for remote sensing imagery. TL allows one to bootstrap a DCNN while preserving the deep visual feature extraction learned over an image corpus from a different image domain. Data augmentation exploits various aspects of remote sensing imagery to dramatically expand small training image data sets and improve DCNN robustness for remote sensing image data. Here, we apply these techniques to the well-known UC Merced data set to achieve the land-cover classification accuracies of 97.8 ± 2.3%, 97.6 ± 2.6%, and 98.5 ± 1.4% with CaffeNet, GoogLeNet, and ResNet, respectively.

Deep convolutional neural networks (DCNNs) have recently emerged as the highest performing approach for a number of image classification applications, including automated land cover classification of high-resolution remote-sensing imagery. In this letter, we investigate a variety of fusion techniques to blend multiple DCNN land cover classifiers into a single aggregate classifier. While feature-level fusion is widely used with deep neural networks, our approach instead focuses on fusion at the classification/information level. Herein, we train three different DCNNs: CaffeNet, GoogLeNet, and ResNet50. The effectiveness of various information fusion methods, including voting, weighted averages, and fuzzy integrals, is then evaluated. In particular, we used DCNN cross-validation results for the input densities of fuzzy integrals followed by evolutionary optimization. This novel approach produces the state-of-the-art classification results up to 99.3% for the UC Merced data set and the 99.2% for the RSD data set.

Accurate land cover classification and detection of objects in high-resolution electro-optical remote sensing imagery (RSI) have long been a challenging task. Recently, important new benchmark data sets have been released which are suitable for land cover classification and object detection research. Here, we present state-of-the-art results for four benchmark data sets using a variety of deep convolutional neural networks (DCNN) and multiple network fusion techniques. We achieve 99.70%, 99.66%, 97.74%, and 97.30% classification accuracies on the PatternNet, RSI-CB256, aerial image, and RESISC-45 data sets, respectively, using the Choquet integral with a novel data-driven optimization method presented in this letter. The relative reduction in classification errors achieved by this data driven optimization is 25%-45% compared with the single best DCNN results.

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.

Deforestation poses a critical environmental challenge with far-reaching impacts on climate change, biodiversity, and local communities. As such, detecting and monitoring deforestation are crucial, and recent advancements in deep learning and remote sensing technologies offer a promising solution to this challenge. In this study, we adapt ChangeFormer, a transformer-based framework, to detect deforestation in the Brazilian Amazon, employing the attention mechanism to analyze spatial and temporal patterns in bi-temporal satellite images. To assess the model’s effectiveness, we employed a robust approach to create a deforestation detection dataset, utilizing Sentinel-2 imagery from select conservation areas in the Brazilian Amazon throughout 2020 and 2021. Our dataset comprises 7,734 pairs of bi-temporal image chips with a resolution of 256×256 pixels and 1,406 pairs of image chips with a resolution of 512×512 pixels. The model achieved an overall accuracy of 93% with corresponding F1 score of 90% and IoU score of 82%. These results demonstrate the potential of transformer-based networks for accurate and efficient deforestation detection.

The modern era of machine learning is focused on data-driven solutions. While this has resulted in astonishing leaps in numerous applications, explainability has not witnessed the same growth. The reality is, most machine learning solutions are black boxes. Herein, we focus on data/information fusion in machine learning. Specifically, we explore four eXplainable Artificial Intelligence (XAI) questions relative to Choquet integral; (i) what is the quality of our inputs and their interactions, (ii) how is the information being combined, (iii) what is the quality of our training data (and thus our learned models), and (iv) what trust do we place in an output? Previously, we derived an initial set of indices for (i)–(iv) on the premise of perfect knowledge. Herein, we make XAI more accurate by taking into consideration what the machine learned. A combination of synthetic data and real-world experiments from remote sensing for fusing deep learners in the context of classification are explored. Our approach leads to performance gain, insights into what was learned, and it helps us realize better future solutions.