In the change detection (CD) task, the UNet architecture has achieved superior results. However, due to the inherent limitation of convolution operations, UNet is inadequate in learning global context and long-range spatial relations. Transformers can capture long-range feature dependencies, but the lack of low-level details may result in limited localization capabilities. Therefore, this article proposes an end-to-end encoding–decoding hybrid transformer model for CD, TransUNetCD, which has the advantages of both transformers and UNet. The model encodes the tokenized image patches from the convolutional neural network (CNN) feature map to extract rich global context information. The decoder upsamples the encoded features, connects them with higher-resolution multiscale features through skip connections to learn local–global semantic features, and restores the full spatial resolution of the feature map to achieve precise localization. The model proposed in this article not only solves the problem that redundant information is generated when extracting low-level features under the UNet framework, but also solves the problem that the relationship between each feature layer cannot be fully modeled and the optimal feature difference representation cannot be obtained. On this basis, we introduce a difference enhancement module to generate a difference feature map containing rich change information. By weighting each pixel and selectively aggregating features, the effectiveness of the network and the accuracy of extracting changing features are improved. The results on multiple datasets demonstrate that, compared to state-of-the-art methods, the TransUNetCD can further reduce false alarms and missed alarms, and the edge of the changing area is more accurate. The model has the highest score in each metric than other baseline models and has a robust generalization ability.
Synthetic aperture radar (SAR) satellites produce large quantities of remote sensing images that are unaffected by weather conditions and, therefore, widely used in marine surveillance. However, because of the hysteresis of satellite-ground communication and the massive quantity of remote sensing images, rapid analysis is not possible and real-time information for emergency situations is restricted. To solve this problem, this paper proposes an on-board ship detection scheme that is based on the traditional constant false alarm rate (CFAR) method and lightweight deep learning. This scheme can be used by the SAR satellite on-board computing platform to achieve near real-time image processing and data transmission. First, we use CFAR to conduct the initial ship detection and then apply the You Only Look Once version 4 (YOLOv4) method to obtain more accurate final results. We built a ground verification system to assess the feasibility of our scheme. With the help of the embedded Graphic Processing Unit (GPU) with high integration, our method achieved 85.9% precision for the experimental data, and the experimental results showed that the processing time was nearly half that required by traditional methods.
In the application of change detection satellite remote sensing images, synthetic aperture radar (SAR) images have become a more important data source. This paper proposes a new end-to-end SAR image change network architecture—TransUNet++SAR—that combines Transformer with UNet++. First, the convolutional neural network (CNN) was used to obtain the feature maps of the single time SAR images layer by layer. Tokenized image patches were encoded to extract rich global context information. Using improved Transformer for effective modeling of global semantic relations can generate rich contextual feature representations. Then, we used the decoder to upsample the encoded features, connected the encoded multi-scale features with the high-level features by sequential connection to learn the local-global semantic features, recovered the full spatial resolution of the feature map, and achieved accurate localization. In the UNet++ structure, the bitemporal SAR images are composed of two single networks, which have shared weights to learn the features of the single temporal image layer by layer to avoid the influence of SAR image noise and pseudo-change on the deep learning process. The experiment results show that the experimental effect of TransUNet++SAR on the Beijing, Guangzhou, and Qingdao datasets were significantly better than other deep learning SAR image change detection algorithms. At the same time, compared with other Transformer related change detection algorithms, the description of the changed area edge was more accurate. In the dataset experiments, the model had higher indices than the other models, especially the Beijing building change datasets, where the IOU was 9.79% higher and F1-score was 4.38% higher.
In traditional image processing, the Fourier transform is often used to transform an image from the spatial domain to the frequency domain, and frequency filters are designed from the perspective of the frequency domain to sharpen or blur the image. In the field of remote sensing change detection, deep learning is beginning to become a mainstream tool. However, deep learning can still refer to traditional methodological ideas. In this paper, we designed a new convolutional neural network (MFGFNet) in which multiple global filters (GFs) are used to capture more information in the frequency domain, thus sharpening the image boundaries and better preserving the edge information of the change region. In addition, in MFGFNet, we use CNNs to extract multi-scale images to enhance the effects and to better focus on information about changes in different sizes (multi-scale combination module). The multiple pairs of enhancements are fused by the difference method and then convolved and concatenated several times to obtain a better difference fusion effect (feature fusion module). In our experiments, the IOUs of our network for the LEVIR-CD, SYSU, and CDD datasets are 0.8322, 0.6780, and 0.9101, respectively, outperforming the state-of-the-art model and providing a new perspective on change detection.
23 February 2024 in IEEE Transactions on Geoscience and Remote Sensing
Semi-supervised change detection is increasingly becoming an interesting and challenging topic for the remote sensing image processing community. As the application of deep learning in change detection becomes more and more widespread, there is a growing lack of labeled training data, which substantially limits the practical application of change detection. In order to discuss a more effective semi-supervised change detection approach and to make more reasonable use of the large amount of remote sensing data, we propose a semi-supervised change detection framework in this paper, which utilizes two different networks to cross-supervise and provide information to each other. Unlike most existing semi-supervised change detection, the proposed framework also incorporates a new filtering algorithm to find better pseudo-labels for the retraining of the two networks in the paper. Then, the computation of the loss functions of the two networks is crossed and the two networks are used for Transformer and CNN different learning paradigms, respectively, while simplifying the classical deep collaborative learning for consistency regularization. In addition, we add two markers to record the highest MIoU of training during retraining, and dynamically update the pseudo-labels as the training metrics progressively improve, which significantly improves the training effect. Our approach is tested on public dataset and achieves very good results that effectively demonstrate the effectiveness of the proposed framework.
18 March 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
On-board processing has become an inevitable choice for the development of remote sensing satellites. For the satellites with dual-line array asynchronous push-broom mode, image registration is crucial for pixel localization and alignment between dual-band strip data. However, due to the high computational complexity of feature detection and feature extraction, it is still a challenge to realize real-time registration for on-board edge devices with limited computing power and power consumption. Therefore, this paper proposes a hardwareoptimized architecture with high-performance and low-energy for on-board registration based on “ARM+FPGA”. On one hand, optimized methods for the image registration algorithm are proposed from both software and hardware perspectives, providing a solution for on-board registration. On the other hand, in the face of the current interface environment of satellites, a design scheme for versatile hardware architecture is proposed. It provides a foundational technical route for hardware deployment of on-board processing. Experimental results show that in this hardware-accelerated architecture, the average acceleration effect of registration algorithm can reach 15 times compared to the unoptimized version. Additionally, the average power consumption is reduced by 60%, and the hardware resource utilization is less than 40%. Importantly, the algorithm's accuracy remains unaffected by these optimizations. The onboard intelligent processing payload deployed with this hardware architecture has been successfully launched and validated in February 2022 in the orbit of the MN200Sar-1 satellite from China. It improves the real-time capability for on-board processing, and aims to achieve a new imaging mechanism where target and information transmission replace strip image transmission.