Given a natural language expression and an image/video, the goal of referring segmentation is to produce the pixel-level masks of the entities described by the subject of the expression. Previous approaches tackle this problem by implicit feature interaction and fusion between visual and linguistic modalities in a one-stage manner. However, human tends to solve the referring problem in a progressive manner based on informative words in the expression, i.e., first roughly locating candidate entities and then distinguishing the target one. In this paper, we propose a cross-modal progressive comprehension (CMPC) scheme to effectively mimic human behaviors and implement it as a CMPC-I (Image) module and a CMPC-V (Video) module to improve referring image and video segmentation models. For image data, our CMPC-I module first employs entity and attribute words to perceive all the related entities that might be considered by the expression. Then, the relational words are adopted to highlight the target entity as well as suppress other irrelevant ones by spatial graph reasoning. For video data, our CMPC-V module further exploits action words based on CMPC-I to highlight the correct entity matched with the action cues by temporal graph reasoning. In addition to the CMPC, we also introduce a simple yet effective Text-Guided Feature Exchange (TGFE) module to integrate the reasoned multimodal features corresponding to different levels in the visual backbone under the guidance of textual information. In this way, multi-level features can communicate with each other and be mutually refined based on the textual context. Combining CMPC-I or CMPC-V with TGFE can form our image or video version referring segmentation frameworks and our frameworks achieve new state-of-the-art performances on four referring image segmentation benchmarks and three referring video segmentation benchmarks respectively. Our code is available at https://github.com/spyflying/CMPC-Refseg .

Hyperspectral (HS) pansharpening intends to synthesize a HS image with a registered panchromatic image, to generate an enhanced image with simultaneous high spectral resolution and high spatial resolution. However, the spectral range gap between the two kinds of images and the need to resolve details for many continuous narrow bands make the technique prone to spectral distortion and spatial blurring. To mitigate the problems, we propose a new HS pansharpening framework via spectrally predictive convolutional neural networks (HyperPNN). In our proposed HyperPNN, spectrally predictive structure is introduced to strengthen the spectral prediction capability of a pansharpening network. Following the concept of the proposed HyperPNN, two specific pansharpening convolutional neural network (CNN) models, i.e., HyperPNN1 and HyperPNN2, are designed. Experimental results from three datasets suggest the excellent performance of our CNN-based HS pansharpening methods.

Infrared remote-sensing images have irreplaceable value in military and civilian research, such as remote surveillance and military reconnaissance. However, under the conditions of complex scenes, infrared ship detection still faces great challenges. Most importantly, because of the limited hardware resource, the spaceborne satellites usually have weak computational processing ability, which makes the traditional convolution neural network (CNN)-based detection algorithms difficult to show their power. In terms of the above facts, this article proposes a high-performance but low-computation and storage-efficient ship detection algorithm to adapt the severe spaceborne environment. Overall, our method contains the following technical steps: 1) we first present a novel iterative precise segmentation of land and sea algorithm to preprocess the complex and diverse remote-sensing scenes; 2) the multivariate Gaussian distribution is then selected to extract the ship target candidate regions to guarantee the detection recall; 3) we adopt the optical panchromatic data to assist the limited infrared data training; and 4) the cascade decision of multisource features including global and local cues is next utilized to gradually eliminate false alarms. Due to the high efficiency, the proposed method can implement well on the hardware platform of DSP and field-programmable gate array (FPGA) architecture. We conduct a series of experiments and compare with the state-of-the-art object detection algorithms. Experimental results show that our method has fewer parameters but can achieve strong detection robustness against the noise, cloud and reef interfere, which verifies the effectiveness of the proposed method.

Single image fog removal is important for surveillance applications and many defogging methods have been proposed, recently. Due to the adverse atmospheric conditions, the scattering properties of foggy images depend on not only the depth information of scene, but also the atmospheric aerosol model, which has more prominent influence on illumination in a fog scene than that in a haze scene. However, recent defogging methods confuse haze and fog, and they fail to consider fully about the scattering properties. Thus, these methods are not sufficient to remove fog effects, especially for images in maritime surveillance. Therefore, this paper proposes a single image defogging method for visual maritime surveillance. Firstly, a comprehensive scattering model is proposed to formulate a fog image in the glow-shaped environmental illumination. Then, an illumination decomposition algorithm is proposed to eliminate the glow effect on the airlight radiance and recover a fog layer, in which objects at the infinite distance have uniform luminance. Secondly, a transmission-map estimation based on the non-local haze-lines prior is utilized to constrain the transmission map into a reasonable range for the input fog image. Finally, the proposed illumination compensation algorithm enables the defogging image to preserve the natural illumination information of the input image. In addition, a fog image dataset is established for visual maritime surveillance. The experimental results based on the established dataset demonstrate that the proposed method can outperform the state-of-the-art methods in terms of both the subjective and objective evaluation criteria. Moreover, the proposed method can effectively remove fog and maintain naturalness for fog images.

Recently, many convolutional neural network (CNN) methods have been designed for hyperspectral image (HSI) classification since CNNs are able to produce good representations of data, which greatly benefits from a huge number of parameters. However, solving such a high-dimensional optimization problem often requires a large number of training samples in order to avoid overfitting. In addition, it is a typical nonconvex problem affected by many local minima and flat regions. To address these problems, in this article, we introduce the naive Gabor networks or Gabor-Nets that, for the first time in the literature, design and learn CNN kernels strictly in the form of Gabor filters, aiming to reduce the number of involved parameters and constrain the solution space and, hence, improve the performances of CNNs. Specifically, we develop an innovative phase-induced Gabor kernel, which is trickily designed to perform the Gabor feature learning via a linear combination of local low-frequency and high-frequency components of data controlled by the kernel phase. With the phase-induced Gabor kernel, the proposed Gabor-Nets gains the ability to automatically adapt to the local harmonic characteristics of the HSI data and, thus, yields more representative harmonic features. Also, this kernel can fulfill the traditional complex-valued Gabor filtering in a real-valued manner, hence making Gabor-Nets easily perform in a usual CNN thread. We evaluated our newly developed Gabor-Nets on three well-known HSIs, suggesting that our proposed Gabor-Nets can significantly improve the performance of CNNs, particularly with a small training set.