Unsupervised domain adaptation (UDA) is an important solution to reduce the bias between the labeled source domain and the unlabeled target domain. It has attracted more attention for optical remote sensing image scene classification and retrieval. Currently, most of the previous work is devoted to closed-set UDA. In fact, the target domain often contains unknown classes. Moreover, some open UDA methods mine structural information of the target domain directly from the type knowledge of the source domain, and less directly from the unlabeled data of the target domain. In this paper, we propose a new self-supervised-driven open-set UDA method combining contrastive self-supervised learning with consistency self-training for optical remote sensing scene classification and retrieval. Specifically, a contrastive self-supervised learning network is introduced to learn discriminative features from the unlabeled target domain data. Moreover, a novel open-set class learning module is developed based on two-level confidence rules and the consistency self-training strategy, which can obtain reliable unknown class samples for co-training. Finally, an open-set dataset including six cross-domain scenarios is constructed based on three public datasets and several experiments are conducted with eleven state-of-the-art domain adaptation methods. Experimental results demonstrate that our proposed method achieves superior performances on the six open-set cross-domain scenarios in both scene classification and retrieval. Especially, our method improves the overall classification accuracies by 9.72% to 24.06% and improve mean average retrieval precisions by 8.06% to 16.21% on the complex UCMD (source domain) → NWPU (target domain) scenario, compared with the other eleven state-of-the-art methods.

Recent advances in deep learning have dramatically improved the performance of content-based remote sensing image retrieval (CBRSIR) with the same distribution of training set (source domain) and test set (target domain). In fact, their distributions are inconsistent in most cases, which can lead to a dramatic decrease in retrieval performance. Currently, some unsupervised domain adaptation (DA) methods for other remote sensing applications have been proposed to eliminate the inconsistency. However, the current unsupervised DA methods do not make full use of the target domain’s distribution characteristics when delineating its decision boundary. This tends to degrade the cross-domain retrieval performance. In this paper, a pseudo-label consistency learning-based unsupervised DA method (PCLUDA) is proposed for cross-domain CBRSIR. Our PCLUDA method minimizes the difference in probability distribution between the target domain and its perturbed output by a pseudo-label self-training and consistency regularization strategy, followed by adjusting the target domain’s decision boundaries to the low-density region. Besides, minimize class confusion is introduced to reduce negative transfer caused by large intra-class variance of remote sensing images. Two cross-domain datasets with twelve cross-domain scenarios are constructed based on six open access datasets to measure DA methods. Experimental results show that our PCLUDA method achieves superior retrieval performances with average retrieval precision improvement by 4.9%-32.3% compared with eight state-of-the-art DA approaches in complex cross-domain scenarios. Furthermore, other experimental results indicate that our PCLUDA can also reach optimal retrieval performances in different kinds of deep learning networks (i.e., vision transformer and convolutional neural networks).

The discriminative ability of image features plays a decisive role in content-based remote sensing image retrieval (CBRSIR). However, the widely-used convolutional neural networks cannot focus on the discriminative features of important scenes, resulting in unsatisfactory retrieval performance in complex contexts. In this article, an attention-enhanced end-to-end discriminative network with multiscale learning for CBRSIR is proposed to solve this issue. First, a multiscale dilated convolution module is embedded into some of ResNet50’s residual blocks to increase the perceptual field and capture the multiscale features of remote sensing image scenes. Then, a lightweight and efficient triplet attention module is added behind each residual block to capture the salient features of remote sensing images and establish the interdimensional dependencies using residual transform. In addition, the end-to-end training approach is performed using an online label smoothing loss to reduce the intraclass variance of features and enhance interclass differentiability. Experimental results on four publicly available remote sensing image datasets show that our network achieves state-of-the-art or competitive performance, especially on complex scene dataset UCMD with an average retrieval precision improvement of 3.23% to 29.35% compared to other new methods.

Convolutional neural networks (CNNs) have proved to achieve state-of-the-art performance in content-based remote sensing image retrieval (CBRSIR). However, CNNs cannot focus on discriminative features of important objects, resulting in unsatisfactory retrieval performance with complex backgrounds and small objects. We therefore propose a multi-attention fusion network with dilated convolution and label smoothing for CBRSIR. First, a dilated convolutional layer is used to replace the fifth convolutional layer in the network to obtain a large receptive field. Then, a contextual transformer attention (CoT) and an efficient channel attention (ECA) are fused together for spatial-wise and channel-wise discriminative features, respectively. The multi-attention module is embedded between the newly added dilated convolutional layer and the followed average pooling layer. Besides, in order to enhance the differences between the discriminative features of those correct and incorrect classes, label smoothing is used to replace the cross-entropy loss function. Some ablation experiments are conducted on six benchmark datasets. Compared to the baseline AlextNet with the above different modules, the mAP values of the proposed network improve by 14.84% to 24.21%. The results indicate that our network can significantly improve the retrieval performance. In addition, we have also conducted some experiments for network migration and comparison with some recent methods (e.g. a triplet deep metric learning and deep feature learning with latent relationship embedding network). Experimental results illustrate that our network can be effectively migrated to other similar CNN models and can achieve state-of-the-art or competitive results.