AbstractDeep learning has shown great power in processing remote sensing data, especially for fine-grained remote sensing ship image classification. However, the lack of a large amount of effective training data greatly limits the performance of neural networks. Based on current data augmentation methods, images of ships on the sea generated for remote sensing have the problem of distortion, blurring, and poor diversity. To tackle this problem, we propose a novel progressive remote sensing ship image data augmentation method that combines ship simulation samples and a neural style transfer (NST) based network to generate a large amount of transferred remote sensing ship images. Our method consists of two stages. The first stage uses a visible light imaging simulation system (VLISS) to generate ship simulation samples through 3D models of real images. This stage can significantly increase the diversity of the training dataset. For the second stage, to eliminate the domain gap between real ship images and ship simulation samples, a few real images and a newly designed NST-based network called Sim2RealNet are employed to realize style transfer from simulation samples to real images. The proposed method was applied to a variety of ship targets to verify its effectiveness compared to other data augmentation methods on remote sensing image classification tasks. The experimental results demonstrate the effectiveness of the proposed method. Index TermsDomain gap, image classification, neural style transfer (NST), remote sensing, ship simulation samples.

The availability of many commercial satellites has created favorable conditions for tracking typical objects in remote sensing sequences, making them widely useful in numerous applications. However, small objects, multiple similar disruptors, background clutter, and occlusion are significant challenges to this field. This study proposes the novel tracker-temporal motion compensation Siamese network (Siam-TMC) for remote sensing tracking. Our method relies on a multidimensional information-aware module and a temporal motion compensation mechanism. Notably, we propose a dual branch-based Dim-Aware module that brings together foreground and high frequency information to distinguish between critical small objects and interferers. In addition, a TMComp mechanism using temporal motion information was designed to mitigate object trajectory drift through the supervision of occlusion detection. Detailed experimental comparisons on a benchmark dataset show that our method outperforms the state-of-the-art tracking models, particularly in occlusion scenarios.

Aircraft classfiers in remotely sensed images based on deep convolutional neural networks play a significant role in military. However, in practical applications, there is a lack of remote sensing fine-grained aircraft data. In this study, we demonstrate that few-shot learning can be effectively used for fine-grained identification of aircraft and propose a new classifier-Adaptive Earth Mover’s Distance (Adap-EMD) for recognition of few-sample fine-grained aircraft. Adap-EMD consists of an efficient block attention mechanism and an adaptive feature measurement filter. The efficient block attention mechanism effectively fuses channel and spatial correlation to capture global features with more pixel-wise relevance and contextual information. The non-parametric adaptive feature measurement filter expresses the key information from the adapted emphasize feature map to achieve a more accurate similarity measurement. Our model outperforms state-of-the-art models on a major few-shot aircraft fine-grained recognition benchmark dataset, introducing only a few additional computations.

Remote sensing image ship fine-grained classification is a challenging vision problem due to factors such as inter-class similarity, real-world image scarcity, and class imbalance. Data augmentation aims to solve these problems from the data perspective. Most of these methods cannot work effectively in complex scenes, which restricts their practical application. Here, we propose a novel data augmentation pipeline based on local-aware image translation to achieve the representation mapping between cross-domain corresponding instances. Our pipeline contains three modules: Imaging Simulation System (ISS), Local-aware Progressive Image-to-Image Translation (LoPIT), and Image Harmonization (IH) modules. The ISS module generates simulated images with correct appearance and diverse features based on the input requirement information. To tackle the domain gap between simulated image and real-world image, we propose the Local-aware CycleGAN in the LoPIT module to achieve mapping based on local-aware learning and apply two sub-modules to progressively complete the remote sensing image global cross-domain translation. The IH module uses image harmonization technology to coordinate the visual appearance between the foreground and background to generate sufficient remote sensing ship images with precise representation, photorealistic style, and harmonious features. Moreover, we present a mixed dataset including real-world images and our synthetic images for remote sensing image fine-grained ship classification. Our dataset named RSSA-12 contains 12 categories of ship targets in 3831 images with high-quality annotated category labels, effectively alleviating the long-tail problem of existing datasets. Experimental results demonstrate that our progressive pipeline outperforms the state-of-the-art data augmentation method on the remote sensing fine-grained ship classification task.

Hyperspectral images (HSIs) are widely used as data formats for remote sensing. Correspondingly, the spectral super-resolution (SSR) technique used to generate high-spatial-resolution HSIs from high-spatial-resolution remote sensing multispectral images (MSIs) has emerged as a popular research topic owing to its high costs and hardware requirements. Generally, existing SSR methods obtain an HSI from the MSI of natural scenes. However, these methods barely learn the complex spectra found in remote sensing images and lack effective treatments for the degradation phenomenon in remote sensing scenes. In this study, a spectral-learning-based transformer network composed of a spectral-response-function (SRF)-guided multilevel feature extraction module (MFEM) and spectral nonlinear mapping learning module (NMLM) is cascaded. The NMLM uses different blocks to gradually learn spatial and spectral information from remote sensing images. Additionally, we focus on atmospheric effects and other factors that influence the generation of remote sensing images and design an MFEM to eliminate these effects. To augment the spectral dimension, an SRF is precisely added to the MFEM as a guide. Experimental results on the Obita Hyperspectral Satellites, Pavia Center, and Washington DC Mall datasets reveal that our method outperforms other state-of-the-art methods in terms of the root mean square error, mean relative absolute error, and relative root mean square error.

The effect of the atmosphere on the propagating energy during the remote sensing imaging simulation is one of the most critical factors affecting the quality of image. The classical methods retrieving atmospheric optical parameters (AOP) have shortcomings in GPU-based imaging simulation application. This paper proposed a method based on deep neural networks (DNN) and principal component analysis (PCA) to compute AOP. Firstly, MODTRAN is employed to obtain large numbers of AOP as original spectrum set in different weather and observation geometry conditions. Then the dimension of original spectrum is reduced by PCA. Next, a DNN is constructed and trained using compressed spectral signatures. Finally, estimated AOP are obtained through inverse PCA by decompressing the output of DNN. The results show that original AOP and estimated AOP have a high spectral similarity which relative error is less than 2%. Compared with the classical methods, DNN can be used to accurately and fast compute AOP with any kind of conditions in remote sensing imaging applications, without consuming large of graphic memory.