Advances in space-based ocean surveillance systems have improved the detection of objects from high-quality remote-sensing big data. Previous studies mainly focused on finding and recognizing objects based on deep learning and statistical frameworks. Studies have not fully explored the transparent and reasonable decision-making process for the final predicted results, which is vital for civil and military applications. An explainable attention network for fine-grained ship image classification is proposed in the present study to bridge this gap. The present study seeks to increase attention to objects’ discriminative parts and explore intrinsic relationships between multiple attention parts and predicted outcomes. Several causal multi-attention maps are generated by combining the multi-head attention mechanism and structural causal model. The convolutional filters in the last layer of the network are divided into several groups, and each group is designed to express specific semantic information under supervision of the filter loss function. The results show which parts of the objects are adopted as the key factors for the network to make the final predicted outcome. In the training process, the network is designed to rapidly focus on the salient feature of objects and played a role in guiding other parts of the network to improve the explainable capability of the network without affecting the discrimination power or compromising the classification accuracy. Extensive experiments based on two public datasets show that the network is highly effective as indicated by high classification accuracy and explainable ability.

The content-based remote sensing image retrieval (CBRSIR) has recently become a hot topic due to its wide applications in analysis of remote sensing data. However, since conventional CBRSIR is unsuitable in harsh environments, this paper focuses on the cross-modality CBRSIR (CM-CBRSIR) between synthetic aperture radar (SAR) and optical images. Besides the large inter-class and small intra-class in CBRSIR, CM-CBRSIR is limited by prominent modality discrepancy caused by different imaging mechanisms. To address this limitation, this study proposes a deep cross-modality hashing network (DCMHN). First, we transform optical images with three channels into four different types of single-channel images to increase diversity of the training modalities. This helps the network to mainly focus on extracting the contour and texture shared features, and make it less sensitive to colour information for images across modalities. Secondly, we combine any type of randomly selected transformed images and its corresponding SAR or optical images to form image pairs that are fed into the networks. The training strategy, with paired image data, eliminates the large cross-modality variations caused by different modalities. Finally, the triplet loss, in combination with the hash function, helps the modal to extract the discriminative features of images and upgrade the retrieval efficiency. To further evaluate the proposed modality, we construct a SAR-optical dual-modality remote sensing image dataset (SODMRSID) containing twelve categories. Experimental results demonstrate the superiority of the proposed method with regards to efficiency and generality.

With the increasing number of remote sensing ship images, it’s vitally important to search for the ship objects that users are interested in from the remote sensing image big data. The existing works are focused on single-modality remote sensing ship image processing. But there is no method of retrieving ship images from different remote sensing image modalities. Besides, the models in existing works only output a predicted result without offering a reasonable explanation. In this work, we propose an interpretable fusion siamese network (IFSN) for addressing the multi-modality remote sensing ship image retrieval (MRSSIR). 1) An interpretable attention feature representation module is proposed to generate multiple attention maps and aggregate the filters of the last convolutional layer, which can focus on the ship’s discriminative parts and make each divided convolutional filter group express specific visual information. 2) A multi-modality correlation learning module is proposed to overcome the intra-modality and inter-modality variations by designing some constraints. 3) A discriminative region mining module is proposed to exhaustively explore all the ship’s discriminative parts available for decision-making of the proposed network. We construct a multi-modality remote sensing ship images dataset (MRSSID) to evaluate the performance of the proposed IFSN. The experimental results exhibit that our IFSN outperforms the existing methods in retrieval accuracy and provides reasonable and intuitive interpretations for the retrieval results.

Nowadays, several remote sensing image capturing technologies are used ranging from unmanned aerial vehicles (UAV) to satellites. Powerful learning-based discriminative features play an essential role in content-based remote sensing image retrieval (CBRSIR). Cross-source CBRSIR (CS-CBRSIR) is used to find relevant remote sensing images across different remote sensing sources (i.e., multispectral images and panchromatic images). But it is limited by large cross-source and intra-source variations caused by different semantic objects, spatial resolution, and spectral resolution. The main limitation of CS-CBRSIR is that it cannot address the inconsistency between different sources and exploit the intrinsic relation between them. This study proposes a discriminative distillation network for CS-CBRSIR to address this limitation. To enlarge the inter-class variations and reduce the intra-class differences, the discriminative features from the first source are first extracted with a well-designed joint optimization configuration (JOC) on the basis of deep neural networks (DNNs). Thereafter, the features extracted from the first source are used as a supervision signal for the second source, feature distribution in common feature space between the first and second sources are made significantly similar. The method proposed in this study simultaneously handles the cross-source and inter-source variations, unlike the existing methods. Extensive experiments on the DSRSID dataset with Euclidean distance verify the effectiveness of our proposed method.method is superior to the existing CS-CBRSIR approaches.

Track interruption will cause serious interference to target tracking, situation awareness, and information fusion. And it is often triggered by a variety of reasons, such as target maneuver, the blind spot of Doppler radar, and excessive long sampling interval, among other causes. All the existing algorithms are based on the hypothetical target motion models and adopt the theory of statistical estimation to measure the similarity between predicted tracks to associate, which have some disadvantages such as unreasonable hypotheses, inapplicable models, and uncertain thresholds. In order to address these problems, in this article, we propose a track segment association dual contrast neural network (TSADCNN) to complete track segment association tasks. First, based on deep metric learning, we propose a temporal and spatial information extraction module to extract track information, especially symmetric correlation information and raise dimension. Second, by constructing an appropriate optimization function, TSADCNN makes track segments belonging to the same target closer, track segments belonging to different targets farther in high-dimensional space. Finally, TSADCNN will pick the nearest neighbor vectors in high-dimensional space as associated tracks. Extensive experiments on simulation data are conducted to evaluate the effectiveness of TSADCNN, and the results demonstrate that this method has good adaptability and anti-noise ability. Furthermore, the contrast experiment shows that it can outperform other TSA methods in both quality and efficiency. Compared with the best performance, the average true association rate of TSADCNN can be increased by 1.3% in long interrupt intervals.

In light of the escalating volume of high-quality remote sensing ship images, the imperative task is to effectively classify and retrieve such images from extensive remote sensing archives. While prior research has yielded promising outcomes in ship classification, a comprehensive framework catering to both ship classification and retrieval remains absent. Additionally, prevailing studies neglect the critical aspect of model interpretability, merely furnishing predicted results devoid of a transparent reasoning process. This opacity, coupled with the high-stakes nature of outcomes, significantly impedes the safe utilization of these models. To address these problems, this paper introduces the cognitive network (Cog-Net), an inherently interpretable model tailored for fine-grained ship classification and retrieval in remote sensing images. Cog-Net imitates the reasoning process employed by domain experts, navigating from perception to cognition during decision-making. The initial stage incorporates the Causal Multi-grained Feature Learning (CMFL) module, mirroring the human perceptual process by identifying salient regions of a ship object within entire images as references for visual concept learning. Subsequently, the second stage introduces the Visual Concept Learning (VCL) module, imitates the human cognitive process by learning basis visual concepts for each ship category and generating predictions through interpretable reasoning based on these basis visual concepts. Furthermore, to facilitate experimentation, a novel dataset, Fine-grained Ship Remote Sensing Image Slices (FGSRSI-23), comprising 23 fine-grained ship subcategories, is constructed. Extensive experiments are conducted, encompassing our FGSRSI-23 dataset alongside two publicly available datasets, FGSC-23 and FGSCR-42. Results attest to the competitiveness of Cog-Net in both ship image classification and retrieval tasks, offering a transparent and interpretable reasoning process for predicted outcomes.