The existence of shadows in very high-resolution panchromatic satellite images can occlude some objects to cause the reduction or loss of their information, particularly in urban scenes. To recover the occluded information of objects, shadow removal is a significant processing procedure for the image interpretation and application. In this paper, we propose a novel framework of shadow detection and removal for panchromatic satellite images to restore the obscured object information. In shadow detection, we present an automatic soft shadow detection method by the combined application of a bimodal histogram splitting method and image matting technique. Soft detection results can exhibit both umbra areas and penumbra areas to describe the shadow distribution precisely. In shadow removal, we propose a spatial adaptive nonlocal sparse shadow removal method to operate at two levels. For the initial step, we apply the line correction method to enhance shadow areas roughly in global. In the refined process, we study the characteristics of objects and shadows, and analyze spatial relationship between them. The second linear radiometric correction and nonlocal sparse model are used to simultaneously control the brightness and smoothness of the recovered shadow areas to be the same as the corresponding nonshadow areas based on group matrix with similar patches. Our method can restore the uniform objects in the shadow areas. High-resolution panchromatic shadow images of different cases and different satellites are processed by the proposed method. The experimental results verify the effectiveness and superiority of the proposed method by comparing three other methods.

Although the RGB image has a high spatial resolution, it only depicts color intensities in red, green, and blue channels, which easily leads to the failure of the tracker based on RGB modality in some challenging scenarios, for example, when the color of the object and background is similar. The hyperspectral image with rich spectral information is more robust in these difficult situations, so it is essential to explore how to effectively apply hyperspectral features to supplement RGB information in object tracking. However, there is no fusion tracking algorithm based on hyperspectral and RGB data. Based on this, we propose a novel fusion tracking framework of hyperspectral and RGB in this article, termed as Transformer-based Fusion Tracking Network (TFTN), to enhance the performance of object tracking. Within the framework, we construct a dual-branch structure based on the Siamese Network to obtain the modality-specific representations of different modality images. Besides, the framework is generic, which is suitable for the Siamese series of tracking algorithms. In addition, we design a Siamese three-dimensional convolutional neural network as the specific branch of hyperspectral modality for synchronous extraction of the spatial and spectral features of hyperspectral data, to give full play to the role of hyperspectral data in improving network tracking performance. Particularly, inspired by the structure of Transformer, we design a Transformer-based fusion module to capture the potential interaction of intra-modality and inter-modality features of different modalities. This is the first work that combines the information of hyperspectral and RGB modalities to improve tracking performance. At the same time, it is also the first time that employs the self-attention module of Transformer to combine the information of different modalities for multi-modality fusion tracking. Experimental results on the dataset composed of hyperspectral and RGB image sequences show that the proposed TFTN tracker is superior to the state-of-the-art trackers, demonstrating the effectiveness of this method.

As a significant means of Earth observation, change detection in high-resolution remote sensing images has received extensive attention. Nevertheless, the variability in imaging conditions introduces style discrepancies and a range of pseudo change regions between bi-temporal image pairs. Furthermore, changing objects possess diverse morphological representations, which makes accurately identifying change areas and delineating their boundaries within complex object distributions increasingly difficult. In response to the aforementioned challenges, we propose Fourier feature interaction and multi-scale perception (FIMP) model for effective change detection. To mitigate the impact of style discrepancies, FIMP employs the Fourier transform to adaptively filter bi-temporal features in the frequency domain, whilst mining the optimized bi-temporal features relevant to the change detection task. To enhance the ability to recognize multi-scale changing objects, FIMP aggregates and emphasizes the change areas with the introduced temporal change enhancement module (TCEM). By utilizing the U-fusion change perception module (UCPM) to perform multi-level bidirectional fusion of change features at different scales, FIMP can further enhance the ability to delineate complex semantic change boundaries. Experiments on three public datasets shows that our approach outperforms seven state-of-the-art methods.

Ship detection is one of the main problems of satellite image analysis. Since ships are scattered on the sea and major ports, large-area remote-sensing images need to be processed in order to realize the detection of ships. In addition, since the satellite is a top-down view, the ship with aspect ratios cannot be covered in complex backgrounds by a horizontal bounding box very well and need a rotating bounding box to achieve this task. Although considerable progress has been made in object detection techniques, there are still challenges for fast detection of ships in large-area remote-sensing images. In this letter, an arbitrary-oriented detector for large-area remote-sensing images is proposed to quickly locate ship positions. A new feature extraction network DCNDarknet25 based on you only look once (YOLO) is designed by reducing paraments and adding deformable convolution (DCN) to improve the speed and accuracy. And the rotation detection capability without angle regression is added to the YOLO detection algorithm for the first time. Finally, thanks to the advantages of our fully convolutional lightweight network, a method for detecting large-area remote-sensing images at once is proposed. In the public dataset HRSC2016 and our own large-area remote-sensing (LARS) image dataset, it has achieved very good accuracy and several times the speed of other algorithms.

At present, hyperspectral image classification (HSIC) technology has been warmly concerned in all walks of life, especially in agriculture. However, existing classification methods operate under the closed-set assumption, which deviates from the real world with open properties. At the same time, there are more serious phenomena of different crops with similar spectrums and the same crops with different spectrum in agricultural hyperspectral data, which is also a great challenge to existing methods. In this work, a fractional Fourier-based frequency-spatial–spectral prototype network (FrFSSPN) is proposed to address the challenges of open-set HSIC in agricultural scenarios. First, fractional Fourier transform (FrFT) is introduced into the network to combine the information in the frequency domain with the spatial–spectral information, so as to expand the difference between different classes on the premise of ensuring the similarity between classes. Then, the prototype learning strategy is introduced into the network to improve the feature recognition capability of the network through prototype loss. Finally, in order to break the stubbornly closed-set property of the closed-set classification (CSC) method, the open-set recognition module is proposed. The difference between the prototype vector and the feature vector is used to judge the unknown class. Experiments on three agricultural hyperspectral datasets show that this method can effectively identify unknown classes without sacrificing the classification accuracy of closed-set, and has satisfactory classification performance.

In this paper, we propose a data augmentation method for ship detection. Inshore ship detection using optical remote sensing imaging is a challenging task owing to an insufficient number of training samples. Although the multilayered neural network method has achieved excellent results in recent research, a large number of training samples is indispensable to guarantee the accuracy and robustness of ship detection. The majority of researchers adopt such strategies as clipping, scaling, color transformation, and flipping to enhance the samples. Nevertheless, these methods do not essentially increase the quality of the dataset. A novel data augmentation strategy was thus proposed in this study by using simulated remote sensing ship images to augment the positive training samples. The simulated images are generated by true background images and three-dimensional models on the same scale as real ships. A faster region-based convolutional neural network (Faster R-CNN) based on Res101netwok was trained by the dataset, which is composed of both simulated and true images. A series of experiments is designed under small sample conditions; the experimental results show that better detection is obtained with our data augmentation strategy.