Multifarious hyperspectral image (HSI) classification methods based on convolutional neural networks (CNN) have been gradually proposed and achieve a promising classification performance. However, hyperspectral image classification still suffers from various challenges, including abundant redundant information, insufficient spectral-spatial representation, irregular class distribution, and so forth. To address these issues, we propose a novel 2D-3D CNN with spectral-spatial multi-scale feature fusion for hyperspectral image classification, which consists of two feature extraction streams, a feature fusion module as well as a classification scheme. First, we employ two diverse backbone modules for feature representation, that is, the spectral feature and the spatial feature extraction streams. The former utilizes a hierarchical feature extraction module to capture multi-scale spectral features, while the latter extracts multi-stage spatial features by introducing a multi-level fusion structure. With these network units, the category attribute information of HSI can be fully excavated. Then, to output more complete and robust information for classification, a multi-scale spectral-spatial-semantic feature fusion module is presented based on a Decomposition-Reconstruction structure. Last of all, we innovate a classification scheme to lift the classification accuracy. Experimental results on three public datasets demonstrate that the proposed method outperforms the state-of-the-art methods.
The detection of an infrared small target faces the problems of background interference and non-obvious target features, which have yet to be efficiently solved. By employing the non-local self-correlation characteristic of the infrared images, the principle component pursuit (PCP)-based methods are demonstrated to be applicable to infrared small target detection in a complex scene. However, existing PCP-based methods heavily depend on the uniform distribution of the background pixels and are prone to generating a high number of false alarms under strong clutter situations. In this paper, we propose a group low-rank regularized principle component pursuit model (GPCP) to solve this problem. First, the local image patches are clustered into several groups that correspond to different grayscale distributions. These patch groups are regularized with a group low-rank constraint, enabling an independent recovery of different background regions. Then, GPCP model integrates the group low-rank components with a global sparse component to extract small targets from the background. Different singular value thresholds can be exploited for image groups corresponding to different brightness and grayscale variance, boosting the recovery of background clutters and also enhancing the detection of small targets. Finally, a customized optimization approach based on alternating direction method of multipliers is proposed to solve this model. We set three representative detection scenes, including the ground background, sea background and sky background for experiment analysis and model comparison. The evaluation results show the proposed model has superiority in background suppression and achieves better adaptability for different scenes compared with various state-of-the-art methods.
30 June 2022 in IEEE Transactions on Instrumentation and Measurement
Infrared and visible image fusion aims to generate an image with prominent target information and abundant texture details. Most existing methods generally rely on manually designing complex fusion rules to realize image fusion. Some deep learning fusion networks tend to ignore the correlation between different level features, which may cause loss of intensity information and texture details in the fused image. To overcome these drawbacks, we propose a multi-level hybrid transmission network for infrared and visible image fusion, which mainly contains the multi-level residual encoder module and the hybrid transmission decoder module. Considering the great difference between infrared and visible images, the multi-level residual encoder module with two independent branches is designed to extract abundant features from source images. To avoid complicated fusion strategies, the concatenate-convolution is applied to fuse features. Towards utilizing information from source images efficiently, the hybrid transmission decoder module is constructed to integrate different level features. Experimental results and analyses on three public datasets demonstrate that our method not only can achieve high quality image fusion but also performs better than comparison methods in terms of qualitative and quantitative comparisons. In addition, the proposed method has good real-time performance in infrared and visible image fusion.
27 August 2024 in IEEE Transactions on Instrumentation and Measurement
The one-shot multiple object tracking framework simultaneously generates detected targets and re-identification (ReID) embeddings, and employs them to associate previous tracks. However, the continuity between the learning and extraction within the ReID task is unconsciously neglected because of treating them as two isolated stages. This leads to unreliable ReID embeddings and poses a challenge to target matching especially in complex and crowded scenarios. To this end, we propose a guided embedding enhancement multiple object tracker, named G2EMOT. It consists of three innovative designs aiming at improving the embedding representation capability for individual targets. First, based on the original feature map, a global instance-specific context decoupling (GICD) module is devised to facilitate the respective feature learning for detection and ReID branches. Then, a heatmap guided embedding enhancement (HGEE) module is introduced to connect the processes of embedding learning and extraction, ensuring that the detected target coordinates are accurately aligned with the discriminative ReID embeddings. Finally, to associate and stabilize matched targets, we present a novel two-gate guided embedding update (TG-GEU) strategy that dynamically updates ReID embeddings. With the proposed three components, G2EMOT achieves outstanding performance on popular MOT benchmarks, while outperforming the existing one-shot tracking methods by a large margin. In particular, it realizes IDF1 of 76.1% on MOT17 and IDF1 of 74.2% on MOT20. The source codes are released at https://github.com/ydhcg-BoBo/G2EMOT.
Convolutional neural networks (CNNs) have demonstrated impressive performance and have been broadly applied in hyperspectral image (HSI) classification. However, two challenging problems still exist: the first challenge is that redundant information is averse to feature learning, which damages the classification performance; the second challenge is that most of the existing classification methods only focus on single-scale feature extraction, resulting in underutilization of information. To resolve the two preceding issues, this article proposes a multiscale cross interaction attention network (MCIANet) for HSI classification. First, an interaction attention module (IAM) is designed to highlight the distinguishability of HSI and dispel redundant information. Then, a multiscale cross feature extraction module (MCFEM) is constructed to detect spectral–spatial features at different scales, convolutional layers, and branches, which can increase the diversity of spectral–spatial features. Finally, we introduce global average pooling to compress multiscale spectral–spatial features and utilize two fully connection layers, two dropout layers to obtain the output classification results. Massive experiments on three benchmark datasets demonstrate the superiority of our presented method compared with the state-of-the-art methods.
Multiple object tracking (MOT) in videos captured by unmanned aerial vehicle (UAV) is a fundamental aspect of computer vision. Recently, the one-shot tracking paradigm integrates the detection and re-identification (ReID) tasks, striking a balance between tracking accuracy and inference speed. This paradigm alleviates task conflicts and achieves remarkable results through various feature decoupling methods. However, in challenging scenarios like drone movements, lighting changes and object occlusion, it still encounters issues with detection failures and identity switches. In addition, traditional feature decoupling methods directly employ channel-based attention to decompose the detection and ReID branches, without a meticulous consideration of the specific requirements of each branch. To address the above problems, we introduce an asymmetric feature enhancement network with a global coordinate-aware enhancement (GCAE) module and an embedding feature aggregation (EFA) module, aiming to optimize the two branches independently. On the one hand, we develop the GCAE module for the detection branch, which effectively merges rich semantic information within the feature space to improve detection accuracy. On the other hand, we introduce the EFA module for the ReID branch, which highlights the significance of pixel-level features and acquires discriminative identity embedding through a local feature aggregation strategy. By efficiently incorporating the GCAE and EFA modules into the one-shot tracking pipeline, we present a novel MOT framework, named AsyUAV. Extensive experiments have demonstrated the effectiveness of our proposed AsyUAV. In particular, it achieves a MOTA of 38.3% and IDF1 of 51.7% on VisDrone2019, and a MOTA of 48.0% and IDF1 of 67.5% on UAVDT, outperforming existing state-of-the-art trackers.
About Me
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences;University of Chinese Academy of Sciences
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences;University of Chinese Academy of Sciences