National Space Science Center, Key Laboratory of Electronics and Information Technology for Space System, Chinese Academy of Sciences, Beijing 100190, China
Auto-generated profile, based on publicly available information
Qi Xiao
National Space Science Center, Key Laboratory of Electronics and Information Technology for Space System, Chinese Academy of Sciences, Beijing 100190, China
AbstractDeep learning has shown great power in processing remote sensing data, especially for fine-grained remote sensing ship image classification. However, the lack of a large amount of effective training data greatly limits the performance of neural networks. Based on current data augmentation methods, images of ships on the sea generated for remote sensing have the problem of distortion, blurring, and poor diversity. To tackle this problem, we propose a novel progressive remote sensing ship image data augmentation method that combines ship simulation samples and a neural style transfer (NST) based network to generate a large amount of transferred remote sensing ship images. Our method consists of two stages. The first stage uses a visible light imaging simulation system (VLISS) to generate ship simulation samples through 3D models of real images. This stage can significantly increase the diversity of the training dataset. For the second stage, to eliminate the domain gap between real ship images and ship simulation samples, a few real images and a newly designed NST-based network called Sim2RealNet are employed to realize style transfer from simulation samples to real images. The proposed method was applied to a variety of ship targets to verify its effectiveness compared to other data augmentation methods on remote sensing image classification tasks. The experimental results demonstrate the effectiveness of the proposed method. Index TermsDomain gap, image classification, neural style transfer (NST), remote sensing, ship simulation samples.
3 October 2022 in IEEE Transactions on Geoscience and Remote Sensing
Remote sensing image ship fine-grained classification is a challenging vision problem due to factors such as inter-class similarity, real-world image scarcity, and class imbalance. Data augmentation aims to solve these problems from the data perspective. Most of these methods cannot work effectively in complex scenes, which restricts their practical application. Here, we propose a novel data augmentation pipeline based on local-aware image translation to achieve the representation mapping between cross-domain corresponding instances. Our pipeline contains three modules: Imaging Simulation System (ISS), Local-aware Progressive Image-to-Image Translation (LoPIT), and Image Harmonization (IH) modules. The ISS module generates simulated images with correct appearance and diverse features based on the input requirement information. To tackle the domain gap between simulated image and real-world image, we propose the Local-aware CycleGAN in the LoPIT module to achieve mapping based on local-aware learning and apply two sub-modules to progressively complete the remote sensing image global cross-domain translation. The IH module uses image harmonization technology to coordinate the visual appearance between the foreground and background to generate sufficient remote sensing ship images with precise representation, photorealistic style, and harmonious features. Moreover, we present a mixed dataset including real-world images and our synthetic images for remote sensing image fine-grained ship classification. Our dataset named RSSA-12 contains 12 categories of ship targets in 3831 images with high-quality annotated category labels, effectively alleviating the long-tail problem of existing datasets. Experimental results demonstrate that our progressive pipeline outperforms the state-of-the-art data augmentation method on the remote sensing fine-grained ship classification task.
To address the problem of intelligent recognition of optical ship targets under low-altitude squint detection, we propose an intelligent recognition method based on simulation samples. This method comprehensively considers geometric and spectral characteristics of ship targets and ocean background and performs full link modeling combined with the squint detection atmospheric transmission model. It also generates and expands squint multi-angle imaging simulation samples of ship targets in the visible light band using the expanded sample type to perform feature analysis and modification on SqueezeNet. Shallow and deeper features are combined to improve the accuracy of feature recognition. The experimental results demonstrate that using simulation samples to expand the training set can improve the performance of the traditional k-nearest neighbors algorithm and modified SqueezeNet. For the classification of specific ship target types, a mixed-scene dataset expanded with simulation samples was used for training. The classification accuracy of the modified SqueezeNet was 91.85%. These results verify the effectiveness of the proposed method.
Temperature field calculation is an important step in infrared image simulation. However, the existing solutions, such as heat conduction modelling and pre-generated lookup tables based on temperature calculation tools, are difficult to meet the requirements of high-performance simulation of infrared images based on three-dimensional scenes under multi-environmental conditions in terms of accuracy, timeliness, and flexibility. In recent years, machine learning-based temperature field prediction methods have been proposed, but these methods only consider the influence of meteorological parameters on the temperature value, while not considering the geometric structure and the thermophysical parameters of the object, which results in the low accuracy. In this paper, a multivariate temperature field prediction network based on heterogeneous data (MTPHNet) is proposed. The network fuses geometry structure, meteorological, and thermophysical parameters to predict temperature. First, a Point Cloud Feature Extraction Module and Environmental Data Mapping Module are used to extract geometric information, thermophysical, and meteorological features. The extracted features are fused by the Data Fusion Module for temperature field prediction. Experiment results show that MTPHNet significantly improves the prediction accuracy of the temperature field. Compared with the v-Support Vector Regression and the combined back-propagation neural network, the mean absolute error and root mean square error of MTPHNet are reduced by at least 23.4% and 27.7%, respectively, while the R-square is increased by at least 5.85%. MTPHNet also achieves good results in multi-target and complex target temperature field prediction tasks. These results validate the effectiveness of the proposed method.