Deep learning-based methods have been widely applied in remote sensing scene classification tasks. Recently, researchers focus more on clarifying the basis of a decision. For example, class activation mapping (CAM) can provide us the evidence by highlighting the related area in an image. However, the interpretability of remote sensing scene classification is more challenging than natural images, since remote sensing images usually contain more complicated objects. As a result, the CAM visual interpretation with traditional convolutional neural networks cannot accurately locate all target objects, which leads to some important objects are ignored. In this letter, we propose a novel model, named encoder-classifier-reconstruction CAM (ECR-CAM) neural network, to provide a more precise visual explanation. Specifically, ECR-CAM consists of four modules: an encoder module, a classifier module, a reconstruction module, and a CAM module. Encoder module is utilized to extract image features, and classifier module accounts for generating predictions. The reconstruction module is the key to locate more target objects. It employs the extracted features to reconstruct the input images, which is a pixel-level process. The reconstruction process allows the features to retain important information about all objects, which cannot be achieved by the classification task alone. Finally, the CAM module can show more target objects with more informative features. Experimental results show that our model not only improves the classification performance but also can locate the target objects more accurately.

Recently, explainable artificial intelligence has received considerable attention. Most existing studies are focusing on the tasks of CNNs-based image classification and RNNs-based time series analysis. In this paper, we pay attention to the more complicated spatiotemporal predictive learning task (SPLT), where both the spatial and temporal information play important roles. To explain the internal mechanism of spatiotemporal prediction models, we propose a comprehensive analysis method. Specifically, with a typical encoder-decoder framework, we focus on two core issues of SPLT: image generation and spatiotemporal dynamics. For the first issue, we develop a quantitative channel perturbation method to explore the importance of features to prediction. Furthermore, we propose a technique called the synthesis of multiple independent components to analyze how these features generate the prediction. According to the experimental results, the coarse- and fine-grained synthesis (CFGS) mechanism is drawn for image generation in SPLT. For the second issue, we propose a state decomposition technique and a state expansion technique to disentangle coupled signals in the spatiotemporal dynamical system. This helps us to explore the mechanism of forming motion. Moreover, to diagnose the movement of a particular region during analysis, we propose a fluorescent stamp-based technique. By observing extensive experimental results, we summarize a collaboration mechanism to explain how the motion is formed in SPLT, namely, the extending the present and erasing the past (EPEP) mechanism. To the best of our knowledge, this is the first work to interpret the internal mechanism of SPLT models.

Cross-modal hashing aims to retrieve similar images from large-scale Earth Observation (EO) data archives, which typically contain multiple satellite sources of remote sensing (RS) images. However, existing cross-modal hashing methods primarily focus on dual-source RS images and often face two main limitations when retrieving multi-source RS images. Firstly, these methods exhibit significant redundancy as they require handling all possible dual-source combinations in multi-source RS images. Secondly, they often rely on pairwise or triplet image sources to construct objective functions, which are not significantly effective in reducing the discrepancies among multiple RS image sources. To address these limitations, we propose a novel Consistency Center-based deep cross-modal Hashing method called C ² Hash for multi-source RS image retrieval. Our C ² Hash employs a multi-branch hashing network to directly encode multi-source RS images into unified hash codes, thereby offering higher processing efficiency. Furthermore, C ² Hash introduces consistency centers to construct a novel objective function. The consistency center represents the shared semantic features among similar multi-source RS images and is generated by a label hashing network. The objective function encourages similar multi-source RS images to approach the same consistency center to align all image sources in a unified Hamming space. Our method can effectively reduce the discrepancies across multiple image sources and generate unified hash codes. To evaluate its effectiveness, we construct a new Multi-Source RS Image dataset called MSRSI, comprising five different types of image sources. We conduct comprehensive experiments to demonstrate the superior performance of our method on the MSRSI dataset. https://github.com/sunyuxi/C2Hash.

Accurate prediction of meteorological elements, such as temperature and relative humidity, is important to human livelihood, early warning of extreme weather, and urban governance. Recently, neural network-based methods have shown impressive performance in this field. However, most of them are overcomplicated and impenetrable. In this paper, we propose a straightforward and interpretable differential framework, where the key lies in explicitly estimating the evolutionary trends. Specifically, three types of trends are exploited. (1) The proximity trend simply uses the most recent changes. It works well for approximately linear evolution. (2) The sequential trend explores the global information, aiming to capture the nonlinear dynamics. Here, we develop an attention-based trend unit to help memorize long-term features. (3) The flow trend is motivated by the nature of evolution, i.e., the heat or substance flows from one region to another. Here, we design a flow-aware attention unit. It can reflect the interactions via performing spatial attention over flow maps. Finally, we develop a trend fusion module to adaptively fuse the above three trends. Extensive experiments on two datasets demonstrate the effectiveness of our method.