Obtaining measured synthetic aperture radar (SAR) data for training automatic target recognition (ATR) models can be too expensive (in terms of time and money) and complex of a process in many situations. In response, researchers have developed methods for creating synthetic SAR data for targets using electro-magnetic prediction software, which is then used to enrich an existing measured training dataset. However, this approach relies on the availability of some amount of measured data. In this work, we focus on the case of having 100% synthetic training data, while testing on only measured data. We use the SAMPLE dataset public released by AFRL, and find significant challenges to learning generalizable representations from the synthetic data due to distributional differences between the two modalities and extremely limited training sample quantities. Using deep learning-based ATR models, we propose data augmentation, model construction, loss function choices, and ensembling techniques to enhance the representation learned from the synthetic data, and ultimately achieved over 95% accuracy on the SAMPLE dataset. We then analyze the functionality of our ATR models using saliency and feature-space investigations and find them to learn a more cohesive representation of the measured and synthetic data. Finally, we evaluate the out-of-library detection performance of our synthetic-only models and find that they are nearly 10% more effective than baseline methods at identifying measured test samples that do not belong to the training class set. Overall, our techniques and their compositions significantly enhance the feasibility of using ATR models trained exclusively on synthetic data.
25 March 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Training deep learning-based synthetic aperture radar automatic target recognition (SAR-ATR) systems for use in an “open-world” operating environment has, thus far proven difficult. Most SAR-ATR systems are designed to achieve maximum accuracy for a limited set of classes, yet ignore the implications of encountering novel target classes during deployment. Even worse, the standard deep learning training objectives fundamentally inherit a closed-world assumption, and provide no guidance for how to handle out-of-distribution (OOD) data. In this work, we develop a novel training procedure called adversarial outlier exposure (AdvOE) to codesign the ATR system for accuracy and OOD detection. Our method introduces a large, diverse, and unlabeled auxiliary training dataset containing samples from the OOD set. The AdvOE objective encourages a deep neural network to learn robust features of the in-distribution training data, while also promoting maximum entropy predictions for adversarially perturbed versions of the OOD data. We experiment with the recent SAMPLE dataset, and find our method nearly doubles the OOD detection performance over the baseline in key settings, and excels when using only synthetic training data. As compared to several other advanced ATR training techniques, AdvOE also affords significant improvements in both classification and detection statistics. Finally, we conduct extensive experiments that measure the effect of OOD set granularity on detection rates; discuss the implications of using different detection algorithms; and develop a novel analysis technique to validate our findings and interpret the OOD detection problem from a new perspective.
27 January 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Recognition systems in the remote sensing domain often operate in “open-world” environments, where they must be capable of accurately classifying data from the in-distribution categories while simultaneously detecting and rejecting anomalous/out-of-distribution (OOD) inputs. However, most modern designs use deep neural networks (DNNs) to perform this recognition function that are trained under “closed-world” assumptions in offline-only environments. As a result, by construction, these systems are ill-posed to handle anomalous inputs and have no mechanism for improving OOD detection abilities during deployment. In this work, we address these weaknesses from two aspects. First, we introduce advanced DNN training methods to codesign for accuracy and OOD detection in the offline training phase. We then propose a novel “learn-online” workflow for updating the DNNs during deployment using a small library of carefully collected samples from the operating environment. To show the efficacy of our methods, we consider experimenting with two popular recognition tasks in remote sensing: scene classification in electro-optical satellite images and automatic target recognition in synthetic aperture radar imagery. In both, we find that our two primary design contributions can individually improve detection performance, while also being complementary. Additionally, we find that detection performance on difficult and highly granular OOD samples can be drastically improved using only tens or hundreds of samples collected from the environment. Finally, through analysis, we determine that the logic for adding/removing samples from the collection library is of key importance and using a proper learning rate during the model update step is critical.
We present advanced Deep Learning (DL) techniques for robust Synthetic Aperture Radar (SAR) automatic target recognition (ATR) in the presence of noise and signal phase errors. Our research focuses on ensuring robust performance of SAR ATR algorithms under noise and adversarial attacks. Robust DL-based SAR ATR is paramount in operational scenarios such as disaster relief, search and rescue, and highly accurate object classification for autonomous vehicles. Our contributions, as described in this paper, include algorithm development and implementation of an advanced deep learning technique known as adversarial training (AT) to mitigate the detrimental effects of sophisticated noise and phase errors. Our research demonstrated that 1) AT improves performance under extended operating conditions, in some cases improving up to 10% over models without AT. 2) The use of AT improves performance when sinusoidal or wideband phase noise is present, in some cases gaining 40% in accuracy that would be lost in the presence of noise. 3) We find the model architecture has significant impact on robustness, with more complex networks showing a greater improvement from AT. 4) The availability of multi-polarization data is always advantageous. To our knowledge no one has provided an extensive analysis of the impact of adversarial machine learning (ML) on SAR image classification. Thus, this paper serves as a comprehensive research revealing the impact of adversarial attack and how to mitigate it.