Deep neural networks (DNNs) have evolved to be the state-of-the-art technique for machine learning tasks. However, their high computational demands make them difficult to deploy on embedded devices with limited hardware resources and strict power budgets. Most embedded systems perform better with 8-bit data processing, prompting extensive research into 8-bit network quantization to enable faster inference. This paper aims to propose a unified 8-bit inference and training framework to support object detection tasks, striving to balance accuracy and speed for conventional convolutional neural networks (CNNs). Initially, this paper establishes a unified full int8 post-training quantization (PTQ) method using KL_divergence to evaluate the range of parameter distributions and thresholds before and after quantization. This method effectively addresses the quantization issues commonly found in networks with linear activations. For networks with nonlinear activations, this paper introduces a hybrid precision post-training quantization (H-PTQ) method that utilizes hybrid precision to perform forward inference, thereby mitigating quantization errors caused by nonlinear activation functions. Furthermore, quantization-aware training (QAT) typically employs the straight-through estimator (STE) for backward propagation of gradients through the quantization function. However, since STE is an approximate computation, this paper proposes an alternative called $\alpha$ -quantization-aware training ( $\alpha$ -QAT). This method replaces the quantized weights in the loss function with affine combinations of the quantized and full-precision weights, enabling more precise forward and backward propagation to fine-tune the errors introduced by quantization. Finally, this paper conducted evaluations of quantized networks on ARM platforms and performed experiments across multiple datasets. The results indicate that the proposed PTQ, H-PTQ, and $\alpha$ -QAT methods achieved maximum accelerations of 4x, 2.3x, and 3.9x, respectively. Additionally, these methods significantly reduced memory overhead by up to 57.11%, 43.16%, and 91.94%, and achieved model compression rates of up to 51.52%, 48.48%, and 49.70%.
15 December 2023 in IEEE Transactions on Aerospace and Electronic Systems
Synthetic aperture radar (SAR) image intelligence interpretation is always a challenging task due to its special imaging mechanism. This paper focuses on the aircraft detection and recognition in large-scale SAR images. The objects in large-size SAR images are usually sparsely distributed and tend to be highly concentrated in certain areas. Therefore, the detection and classification in SAR images are affected easily by imaging environment, leading to ineffective and instability. It suffers from time-consuming, high false-alarm rate, and low recognition rate. To alleviate these problems, a joint probability detector (JPD) integrated with scattering attention (SA) is designed for aircraft detection and classification. Based on the fact that all aircraft are parked in the airport area, an airport detection method with mixed strategy (ADMS) is proposed to obtain the valid region (airport), narrowing downing the scope of detection area, increasing the detection efficiency as well as suppressing a lot of false alarms outside the airport. Considering that it's easy to generate false alarms within the airport on account of clutter scattering interference, a two-stage detector is introduced to improve the accuracy by replacing a stronger region proposal network (RPN) and optimizing a lower bound to a joint probabilistic objective over two stages. Furthermore, the classification encoder module incorporates scattering attention into convolution neural network to leverage the distribution relation among strong scattered points to enhance the classification performance. Finally, extensive experiments on the ZKXT and GaoFen-3 datasets verify the effectiveness and superiority of the proposed approach, and surpass runner up by 5.3% with mF1 indicator. On a large GaoFen3 image (16550× 11945 pixels), the proposed method improves the detection speed by 187.52%, without sacrificing the accuracy. More importantly, the proposed method has been applied to the SAR image interpretation and achieved the expected effect in the actual application.
The rapid development of neural networks has come at the cost of increased computational complexity. Neural networks are both computationally intensive and memory intensive; as such, the minimal energy and computing power of satellites pose a challenge for automatic target recognition (ATR). Knowledge distillation (KD) can distill knowledge from a cumbersome teacher network to a lightweight student network, transferring the essential information learned by the teacher network. Thus, the concept of KD can be used to improve the accuracy of student networks. Even when learning from a teacher network, there is still redundancy in the student network. Traditional networks fix the structure before training, such that training does not improve the situation. This paper proposes a distillation sparsity training (DST) algorithm based on KD and network pruning to address the above limitations. We first improve the accuracy of the student network through KD, and then through network pruning, allowing the student network to learn which connections are essential. DST allows the teacher network to teach the pruned student network directly. The proposed algorithm was tested on the CIFAR-100, MSTAR, and FUSAR-Ship data sets, with a 50% sparsity setting. First, a new loss function for the teacher-pruned student was proposed, and the pruned student network showed a performance close to that of the teacher network. Second, a new sparsity model (uniformity half-pruning UHP) was designed to solve the problem that unstructured pruning does not facilitate the implementation of general-purpose hardware acceleration and storage. Compared with traditional unstructured pruning, UHP can double the speed of neural networks.