Pixel-level classification for very high resolution (VHR) images is a crucial but challenging task in remote sensing. However, since the diverse ways of satellite image acquisition and the distinct structures of various regions, the distributions of the same semantic classes among different data sets are dissimilar. Therefore, the classification model trained on one data set (source domain) may collapse, when it is directly applied to another one (target domain). To solve this problem, many adversarial-based domain adaptation methods have been proposed. However, these methods only consider the source and the target domains independently in the adversarial training, where only the target domain is explicitly contributed to narrow the gap between the distributions of both domains. Unlike previous methods, we propose a triplet adversarial domain adaptation (TriADA) method that jointly considers both domains to learn a domain-invariant classifier by a novel domain similarity discriminator. Specifically, the discriminator takes a triplet of segmentation maps as input, where two segmentation maps from the same domain are to be distinguished from the two maps from the different domains during the adversarial learning. Consequently, it explicitly considers both domains' information to narrow the distribution gap across domains. To enhance the discriminability of the classifier on the target domain, a class-aware self-training strategy, which depends on the output of the discriminator, is proposed to assign pseudo-labels with high adapted confidence on target data to retrain the classifier. Extensive experiments on several VHR pixel-level classification benchmarks demonstrate the effectiveness of our method as well as its superiority to the-state of the art.

Multiimage super-resolution (MISR), as one of the most promising directions in remote sensing, has become a needy technique in the satellite market. A sequence of images collected by satellites often has plenty of views and a long time span, so integrating multiple low-resolution views into a high-resolution image with details emerges as a challenging problem. However, most MISR methods based on deep learning cannot make full use of multiple images. Their fusion modules are incapable of adapting to an image sequence with weak temporal correlations well. To cope with these problems, we propose a novel end-to-end framework called TR-MISR. It consists of three parts: An encoder based on residual blocks, a transformer-based fusion module, and a decoder based on subpixel convolution. Specifically, by rearranging multiple feature maps into vectors, the fusion module can assign dynamic attention to the same area of different satellite images simultaneously. In addition, TR-MISR adopts an additional learnable embedding vector that fuses these vectors to restore the details to the greatest extent. TR-MISR has successfully applied the transformer to MISR tasks for the first time, notably reducing the difficulty of training the transformer by ignoring the spatial relations of image patches. Extensive experiments performed on the PROBA-V Kelvin dataset demonstrate the superiority of the proposed model that provides an effective method for transformers in other low-level vision tasks.

In computer vision, fine-tuning is the de-facto approach to leverage pre-trained vision models to perform downstream tasks. However, deploying it in practice is quite challenging, due to adopting parameter inefficient global update and heavily relying on high-quality downstream data. Recently, prompt-based learning, which adds the task-relevant prompt to adapt the pre-trained models to downstream tasks, has drastically boosted the performance of many natural language downstream tasks. In this work, we extend this notable transfer ability benefited from prompt into vision models as an alternative to fine-tuning. To this end, we propose parameter-efficient Prompt tuning (Pro-tuning) to adapt diverse frozen pre-trained models to a wide variety of downstream vision tasks. The key to Pro-tuning is prompt-based tuning, i.e ., learning task-specific vision prompts for downstream input images with the pre-trained model frozen. By only training a small number of additional parameters, Protuning can generate compact and robust downstream models both for CNN-based and transformer-based network architectures. Comprehensive experiments evidence that the proposed Protuning outperforms fine-tuning on a broad range of vision tasks and scenarios, including image classification (under generic objects, class imbalance, image corruption, natural adversarial examples, and out-of-distribution generalization), and dense prediction tasks such as object detection and semantic segmentation.

Most deep-learning-based vision tasks rely heavily on crowd-labeled data, and a deep neural network (DNN) is usually impacted by the laborious and time-consuming labeling paradigm. Recently, foundation models (FMs) have been presented to learn richer features from multi-modal data. Moreover, a single foundation model enables zero-shot predictions on various vision tasks. The above advantages make foundation models better suited for remote sensing images, where image annotations are more sparse. However, the inherent differences between natural images and remote sensing images hinder the applications of the foundation model. In this context, this paper provides a comprehensive review of common foundation models and domain-specific foundation models for remote sensing, and it summarizes the latest advances in vision foundation models, textually prompted foundation models, visually prompted foundation models, and heterogeneous foundation models. Despite the great potential of foundation models for vision tasks, open challenges concerning data, model, and task impact the performance of remote sensing images and make foundation models far from practical applications. To address open challenges and reduce the performance gap between natural images and remote sensing images, this paper discusses open challenges and suggests potential directions for future advancements.

Ship detection from synthetic aperture radar (SAR) images is inherently subject to the special imaging mechanism of SAR. In recent years, deep-learning-based techniques for detecting objects from optical images have rapidly advanced and promoted the development of SAR image detection technology. However, the strong speckle noise in SAR images degrades low-level feature learning in shallow layers, hindering the higher level learning of semantic features for object detection. In view of the problems encountered in direct end-to-end feature learning for object detection and the close relationship between objects and auxiliary cues, a multitask learning-based object detector (MTL-Det) is proposed in this article to distinguish ships in SAR images. The proposed approach models the ship detection problem, not as a single object detection task, but as three cooperative tasks. The model involves two auxiliary subtasks that are focused on learning object-specific cues (e.g., texture and shape) for the ship detection task, which is constrained by the pseudoground truth generated by the main task. Assisted by auxiliary subtasks, the low-level features are robust to speckle noise and reliably support high-level feature learning. Compared with traditional single-task-based object detectors, more discriminative object-specific features are learned by multitask learning without the extra cost of manual labeling. The experiments conducted in this study help demonstrate the advantages of MTL-Det in improving the ship detection performance on two SAR datasets: high-resolution SAR images dataset and large-scale SAR ship detection dataset-v1.0.

Hashing-based semantic similarity search is becoming increasingly important for building large-scale content-based retrieval system. The state-of-the-art supervised hashing techniques use flexible two-step strategy to learn hash functions. The first step learns binary codes for training data by solving binary optimization problems with millions of variables, thus usually requiring intensive computations. Despite simplicity and efficiency, locality-sensitive hashing (LSH) has never been recognized as a good way to generate such codes due to its poor performance in traditional approximate neighbor search. We claim in this paper that the true merit of LSH lies in transforming the semantic labels to obtain the binary codes, resulting in an effective and efficient two-step hashing framework. Specifically, we developed the locality-sensitive two-step hashing (LS-TSH) that generates the binary codes through LSH rather than any complex optimization technique. Theoretically, with proper assumption, LS-TSH is actually a useful LSH scheme, so that it preserves the label-based semantic similarity and possesses sublinear query complexity for hash lookup. Experimentally, LS-TSH could obtain comparable retrieval accuracy with state of the arts with two to three orders of magnitudes faster training speed.