Single-frame infrared small target detection is still a challenging task due to the complex background and unobvious structural characteristics of small targets. Recently, convolutional neural networks (CNN) began to appear in the field of infrared small target detection and have been widely used for excellent performance. However, existing CNN-based methods mainly focus on local spatial features while ignoring the long-range contextual dependencies between small targets and backgrounds. To capture the global context-aware information, we propose fusion network architecture of transformer and CNN (FTC-Net), which consists of two branches. The CNN-based branch uses a U-Net with skip connections to obtain low-level local details of small targets. The transformer-based branch applies hierarchical self-attention mechanisms to learn long-range contextual dependencies. Specifically, the transformer branch can suppress background interferences and enhance target features. To obtain local and global feature representation, we design a feature fusion module to realize the feature concentration of two branches. We implement ablation and comparative experiments on a publicly accessed SIRST dataset. Experimental results show that the transformer-based branch is effective and suggest the superiority of the proposed FTC-Net compared with other state-of-the-art methods.

With the rapid advances in remote-sensing technologies and the larger number of satellite images, fast and effective object detection plays an important role in understanding and analyzing image information, which could be further applied to civilian and military fields. Recently object detection methods with region-based convolutional neural network have shown excellent performance. However, these two-stage methods contain region proposal generation and object detection procedures, resulting in low computation speed. Because of the expensive manual costs, the quantity of well-annotated aerial images is scarce, which also limits the progress of geospatial object detection in remote sensing. In this paper, on the one hand, we construct and release a large-scale remote-sensing dataset for geospatial object detection (RSD-GOD) that consists of 5 different categories with 18,187 annotated images and 40,990 instances. On the other hand, we design a single shot detection framework with multi-scale feature fusion. The feature maps from different layers are fused together through the up-sampling and concatenation blocks to predict the detection results. High-level features with semantic information and low-level features with fine details are fully explored for detection tasks, especially for small objects. Meanwhile, a soft non-maximum suppression strategy is put into practice to select the final detection results. Extensive experiments have been conducted on two datasets to evaluate the designed network. Results show that the proposed approach achieves a good detection performance and obtains the mean average precision value of 89.0% on a newly constructed RSD-GOD dataset and 83.8% on the Northwestern Polytechnical University very high spatial resolution-10 (NWPU VHR-10) dataset at 18 frames per second (FPS) on a NVIDIA GTX-1080Ti GPU.

Remote sensing image captioning (RSIC) has great significance in image understanding, which describes the image content in natural language. Existing methods are mainly based on deep learning and rely on the encoder–decoder model to generate sentences. In the decoding process, recurrent neural network (RNN) and long short-term memory (LSTM) are normally applied to sequentially generate image captions. In this letter, the transformer encoder–decoder is combined with grid features to improve the RSIC performance. First, the pretrained convolutional neural network (CNN) is used to extract grid-based visual features, which are encoded as vectorial representations. Then, the transformer outputs semantic descriptions to bridge visual features and natural language. Besides, the self-critical sequence training (SCST) strategy is applied to further optimize the image captioning model and improve the quality of generated sentences. Extensive experiments are organized on three public datasets of RSCID, UCM-Captions, and Sydney-Captions. Experimental results demonstrate the effectiveness of SCST strategy and the proposed method achieves superior performance compared with the state-of-the-art image captioning approaches on the RSCID dataset.

The fusion of infrared intensity and polarization images can generate a single image with better visible perception and more vital information. Existing fusion methods based on a convolutional neural network (CNN), with local feature extraction, have the limitation of fully exploiting salient target features of polarization. In this Letter, we propose a transformer-based deep network to improve the performance of infrared polarization image fusion. Compared with existing CNN-based methods, our model can encode long-range features of infrared polarization images to obtain global contextual information using the self-attention mechanism. We also design a loss function with the self-supervised constraint to boost the performance of fusion. Experiments on the public infrared polarization dataset validate the effectiveness of the proposed method. Our approach achieves better fusion performance than the state-of-the-art.

Infrared small target detection (IRSTD) involves accurately locating the positions of small and dim targets in infrared images and serves as a fundamental component in modern sensing and measurement systems, such as maritime vessel search and airborne monitoring. In the spatial domain, background clutter and sensor-induced noise exhibit similar visual characteristics to true targets, leading to false alarms and missed detections. Existing detection methods primarily focus on extracting local spatial features, while overlooking the differences between targets and background in the frequency domain. To address this limitation, we propose a Frequency-Aware Attention Network (FAA-Net) for IRSTD. Specifically, we design a Frequency-Aware Attention Module (FAM) based on discrete cosine transform to capture informative high-frequency components and enhance local contrast. Furthermore, we construct a Transformer and Convolution Fusion Block (TCFB) to combine the global modeling ability of Transformers with the local feature extraction strength of convolutional layers. By applying FAMs and TCFBs to both shallow and deep layers, FAA-Net captures multi-scale semantic and structural features to generate robust target representations. From a measurement perspective, FAA-Net provides reliable pixel-level outputs that correspond to measurable physical quantities, such as target location and area. We conduct experiments on three publicly available datasets, including NUDT-SIRST, NUAA-SIRST, and IRSTD-1K. Experimental results demonstrate that our presented FAA-Net achieves superior performance compared to existing deep learning-based methods in IRSTD. The source code is available at https://github.com/Dr-Zhuang/FAA-Net-IRSTD.

Traditional Synthetic Aperture Radar (SAR) image sea-land segmentation algorithms overlook the ship distribution priori-information provided by the Automatic Identification System (AIS) data, resulting in poor segmentation performance in complex environments such as ports, marine wetlands, beaches, and other sea-land boundaries. To address the above issues, this paper comprehensively uses dual-polarization (VV and VH) SAR images and AIS data as the data source, and it specifically proposes a novel Pyramid Vision Transformer (PVT) assisted by the long-time AIS data (AIS-PVT) for sea-land segmentation. AIS-PVT is the first attempt to integrate the ship distribution density priori-information, provided by the long-time AIS data, into PVT network, thus the multi-scale features of the sea and land can be better distinguished. In the decoding stage, we design a Feature Filtering Module (FFM). It aggregates features separately along two spatial directions from the skip connections, enhancing the representation of objects of interest while reducing the influence of redundant information. Furthermore, we develop a boundary-pixel-aware function to steer the model training process, allowing AIS-PVT to concentrate more on the neighborhood information of boundary pixels. Importantly, the AIS-PVT method captures global multi-scale information and enhances the model’s data fusion capability. The conclusive experimental results demonstrate the superior performance of our approach in sea-land segmentation tasks, outperforming other state-of-the-art techniques.