Although the RGB image has a high spatial resolution, it only depicts color intensities in red, green, and blue channels, which easily leads to the failure of the tracker based on RGB modality in some challenging scenarios, for example, when the color of the object and background is similar. The hyperspectral image with rich spectral information is more robust in these difficult situations, so it is essential to explore how to effectively apply hyperspectral features to supplement RGB information in object tracking. However, there is no fusion tracking algorithm based on hyperspectral and RGB data. Based on this, we propose a novel fusion tracking framework of hyperspectral and RGB in this article, termed as Transformer-based Fusion Tracking Network (TFTN), to enhance the performance of object tracking. Within the framework, we construct a dual-branch structure based on the Siamese Network to obtain the modality-specific representations of different modality images. Besides, the framework is generic, which is suitable for the Siamese series of tracking algorithms. In addition, we design a Siamese three-dimensional convolutional neural network as the specific branch of hyperspectral modality for synchronous extraction of the spatial and spectral features of hyperspectral data, to give full play to the role of hyperspectral data in improving network tracking performance. Particularly, inspired by the structure of Transformer, we design a Transformer-based fusion module to capture the potential interaction of intra-modality and inter-modality features of different modalities. This is the first work that combines the information of hyperspectral and RGB modalities to improve tracking performance. At the same time, it is also the first time that employs the self-attention module of Transformer to combine the information of different modalities for multi-modality fusion tracking. Experimental results on the dataset composed of hyperspectral and RGB image sequences show that the proposed TFTN tracker is superior to the state-of-the-art trackers, demonstrating the effectiveness of this method.

Ship detection is one of the main problems of satellite image analysis. Since ships are scattered on the sea and major ports, large-area remote-sensing images need to be processed in order to realize the detection of ships. In addition, since the satellite is a top-down view, the ship with aspect ratios cannot be covered in complex backgrounds by a horizontal bounding box very well and need a rotating bounding box to achieve this task. Although considerable progress has been made in object detection techniques, there are still challenges for fast detection of ships in large-area remote-sensing images. In this letter, an arbitrary-oriented detector for large-area remote-sensing images is proposed to quickly locate ship positions. A new feature extraction network DCNDarknet25 based on you only look once (YOLO) is designed by reducing paraments and adding deformable convolution (DCN) to improve the speed and accuracy. And the rotation detection capability without angle regression is added to the YOLO detection algorithm for the first time. Finally, thanks to the advantages of our fully convolutional lightweight network, a method for detecting large-area remote-sensing images at once is proposed. In the public dataset HRSC2016 and our own large-area remote-sensing (LARS) image dataset, it has achieved very good accuracy and several times the speed of other algorithms.

In this paper, we propose a data augmentation method for ship detection. Inshore ship detection using optical remote sensing imaging is a challenging task owing to an insufficient number of training samples. Although the multilayered neural network method has achieved excellent results in recent research, a large number of training samples is indispensable to guarantee the accuracy and robustness of ship detection. The majority of researchers adopt such strategies as clipping, scaling, color transformation, and flipping to enhance the samples. Nevertheless, these methods do not essentially increase the quality of the dataset. A novel data augmentation strategy was thus proposed in this study by using simulated remote sensing ship images to augment the positive training samples. The simulated images are generated by true background images and three-dimensional models on the same scale as real ships. A faster region-based convolutional neural network (Faster R-CNN) based on Res101netwok was trained by the dataset, which is composed of both simulated and true images. A series of experiments is designed under small sample conditions; the experimental results show that better detection is obtained with our data augmentation strategy.

Hyperspectral video with spatial and spectral information has great potential to improve object tracking performance. However, the limited hyperspectral training samples hinder the development of hyperspectral object tracking. Since hyperspectral data has multiple bands, from which any three bands can be extracted to form pseudocolor images, we propose a Transformer-based multimodality information transfer network (TMTNet), aiming to improve the tracking performance by efficiently transferring the information of multimodality data composed of RGB and hyperspectral in the hyperspectral tracking process. The multimodality information needed to be transferred mainly includes the RGB and hyperspectral multimodality fusion information and the RGB modality information. Specifically, we construct two subnetworks to transfer the multimodality fusion information and the robust RGB visual information, respectively. Among them, the multimodality fusion information transfer subnetwork is designed based on the dual Siamese branch structure. The subnetwork employs the pretrained RGB tracking model as the RGB branch to guide the training of the hyperspectral branch with little training samples. The RGB modality information transfer subnetwork is designed based on a pretrained RGB tracking model with good performance to improve the tracking network’s generalization and accuracy in unknown complex scenes. In addition, we design an information interaction module based on Transformer in the multimodality fusion information transfer subnetwork. The module can fuse multimodality information by capturing the potential interaction between different modalities. We also add a spatial optimization module to TMTNet, which further optimizes the object position predicted by the subject network by fully retaining and utilizing detailed spatial information. Experimental results on the only available hyperspectral tracking benchmark dataset show that the proposed TMTNet tracker outperforms the advanced trackers, demonstrating the effectiveness of this method.

Ship detection based on synthetic aperture radar (SAR) images has made a breakthrough in recent years. However, small ships, which may be regarded as speckle noise, pose enormous challenges to the accurate detection of SAR images. In order to enhance the detection performance of small ships in SAR images, a novel detection method named a spatial information integration network (SII-Net) is proposed in this paper. First, a channel-location attention mechanism (CLAM) module which extracts position information along with two spatial directions is proposed to enhance the detection ability of the backbone network. Second, a high-level features enhancement module (HLEM) is customized to reduce the loss of small target location information in high-level features via using multiple pooling layers. Third, in the feature fusion stage, a refined branch is presented to distinguish the location information between the target and the surrounding region by highlighting the feature representation of the target. The public datasets LS-SSDD-v1.0, SSDD and SAR-Ship-Dataset are used to conduct ship detection tests. Extensive experiments show that the SII-Net outperforms state-of-the-art small target detectors and achieves the highest detection accuracy, especially when the target size is less than 30 pixels by 30 pixels.

With the development of Earth observation technology, it becomes easier and easier to acquire multi-modal image data at the same time. To improve the performance of multi-modal remote sensing detection algorithm, a new fusion feature optimization detection network (FFODNet) is proposed. The method is designed to solve the problem of performance degradation caused by the unreliability of single modal data in multi-modal remote sensing data. The key to obtain high quality fusion features from multi-modal data with interference is to suppress single modal redundant features and fully integrate multi-modal features. The proposed method mainly includes two improvements. Firstly, a novel joint expression optimization module (JEOM) is designed to enhance the target features and suppress the redundant and interference features that affect the fusion effect. Additionally, we propose a novel specific information enhancement module (SIEM) to further enhance the discriminative feature information of targets within each modal image. Experiments on DroneVehicle dataset show that our proposed method is state-of-the-art on this dataset.