UAV aerial Visible-Infrared (RGBT) object detection has been widely applied in fields such as military operations and rescue missions. However, although numerous UAV aerial RGBT object detection methods exist, several challenges remain in this field. On the one hand, drones typically operate at high altitudes, and objects only occupy a small number of pixels in imaging, posing a significant challenge to object detection. On the other hand, spatial misalignment between modalities remains a major obstacle in cross-modal fusion—especially given the small size of the objects. To address the above issues, this paper proposes IM-CMDet, an intra-modal enhancement and cross-modal fusion network for small object detection in UAV-based RGBT imagery, which comprises three effective modules: the Detail-Semantics Joint Enhancement module (DSJE), the Differential-based Fusion Weight Generation module (DFWG) and the Feature Reconstruction Network (FRN). The DSJE module prevents small object features from being overwhelmed by background noise through optimizing feature representations across different levels. The FRN module is designed to overcome modality differences and build inter-modality information correlation via swin-Transformer architecture. To further enhance the network’s sensitivity to small objects, the DFWG combines differential and spatial attention to generate the final fusion weights while reducing the impact of background noise on detection performance. Extensive experiments on RGBTDronePerson and two additional benchmarks demonstrate that IM-CMDet achieves state-of-the-art performance through effective cross-modal fusion, significantly advancing small-object detection in complex aerial scenarios. The code is available at https://github.com/RS-Minchao/IM-CMDet.
10 February 2022 in IEEE Transactions on Multimedia
In hyperspectral imagery, target detection algorithms are usually based on the spectral signature information. Due to the advance of the spatial resolution of hyperspectral sensors, the ground sample distance may be much smaller than the size of targets. As a result, targets often occupy multiple consecutive pixels, which are referred to as multi-pixel targets. In this paper, we investigate the target detection problem for multi-pixel targets in hyperspectral imagery, when the target spectral signature is known. Jointly exploiting the pixels occupied by a target of interest, we propose a multi-pixel target detector resorting to the generalized likelihood ratio test criterion. Closed-form expressions for the probabilities of the false alarm and detection are derived, which are verified using Monte Carlo simulations. Experimental results on four real hyperspectral datasets show that the proposed detector outperforms its counterparts.
3 March 2023 in IEEE Transactions on Geoscience and Remote Sensing
Recently, a generalized likelihood ratio test (GLRT)-based multipixel target detector for hyperspectral imagery was proposed. With joint exploitation of the pixels occupied by a target of interest, the detection performance was significantly improved. However, it still faces a pixel selection problem in practice. In this paper, we address the pixel selection problem for the multipixel target detector in practice. First, we propose an adaptive target pixel selection method based on spectral similarity and spatial connectivity characteristics. Second, we propose a method to collect the pixels spatially closest to the target pixels as the training background pixels so that their residual background component share the same statistical characteristics with high probability. To exclude potential target pixels in the collected training background pixels, an iterative version of the GLRT-based multipixel target detector is proposed. It is easy to set the key parameters of the proposed method, which is attractive in practice. Experimental results on four real hyperspectral datasets show that the proposed method outperforms its counterparts in terms of detection performance.
27 June 2025 in IEEE Transactions on Neural Networks and Learning Systems
The integration of hyperspectral (HS) imagery and light detection and ranging (LiDAR) data for land cover classification has emerged as a prominent research focus. Despite the satisfactory classification accuracies achieved by existing methodologies, several unaddressed issues that remain warrant consideration. First, current approaches overlook the pronounced spectral and spatial heterogeneities in remote sensing (RS) images designated for multiclassification tasks, limiting the performance of classification models. Moreover, most existing studies amalgamate elevation features with other characteristics through simple addition and interaction operations, and they do not delve deeply into exploiting elevation height information, leading to an imbalance in the representation of elevation height. In light of the aforementioned issues, this article introduces a spatial–spectral heterogeneity-aware network (S2HANet) for the joint classification of HS and LiDAR data. Specifically, a shared spectral correction module (SSCM) is designed in the spectral branch to preliminarily alleviate the problem of large intraclass variance, followed by the use of a contrastive learning framework to enhance the intraclass compactness and interclass separability of spectral features. A multichannel signed distance discrimination module (MCSDDM) is developed to learn the distance relationships between intra- and interclass pixels and boundaries, and using prior boundary information to improve spatial boundary information. In addition, an elevation boost module (EBM) and an elevation injection module (EIM) are meticulously designed to phase-in elevation height information, further enhancing the utilization of elevation data and better facilitating the fusion of the two modalities. The proposed S2HANet has demonstrated exceptional classification performance across three opening benchmark datasets.
10 July 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Coastal wetland monitoring is essential for protecting marine and terrestrial ecosystems. However, the complex spatial, temporal, and spectral characteristics of these wetlands pose significant challenges for accurate mapping. Coastal wetlands exhibit high spatial heterogeneity due to varied landforms and fluctuating hydrological conditions. Temporal dynamics driven by seasonal cycles and tidal effects, along with spectral similarities across categories and variability within categories, further complicate accurate classification. Existing mapping methods struggle to integrate spatiotemporal and spectral information from time-series data, limiting their ability to model complex boundaries and dynamic changes. To address these challenges, we propose a deep temporal-spatial-spectral interaction learning framework for coastal wetland mapping using time-series remote sensing imagery. The model incorporates a multi-scale, multi-dimensional convolutional module to extract and interact spatial, temporal, and spectral features. A hybrid transformer-convolution module enhances fine-grained feature extraction. While a temporal index extraction module and dual-focus attention module provide prior information, improve the accuracy of challenging wetland category identification. The feature fusion and adaptive classification module, dynamically assigning weights based on the importance of temporal, spatial, and spectral features for optimal information aggregation. boosting classification performance. We validated the model using Sentinel-2 time-series multispectral datasets from the Yellow River Estuary, Yancheng coastal wetlands, and Hangzhou Bay. Experimental results demonstrate competitive performance: the overall accuracy (OA) reaches 98.07% for Yellow River Estuary, 91.55% for Yancheng, and 97.82% for Hangzhou Bay.
Pan-sharpening aims to generate high-resolution (HR) multispectral (MS) images by fusing HR panchromatic (PAN) and low-resolution (LR) MS images covering the same area. However, due to the lack of real HR MS reference images, how to accurately evaluate the quality of a fused image without reference is challenging. On the one hand, most methods evaluate the quality of the fused image using the full-reference indices based on the simulated experimental data on the popular Wald’s protocol; however, this remains controversial to the full-resolution data fusion. On the other hand, existing limited no reference methods, most of which depend on manually crafted features, cannot fully capture the sensitive spatial/spectral distortions of the fused image. Therefore, this paper proposes a perceptual quality assessment method based on deep feature similarity measure. The proposed network includes spatial/spectral feature extraction and similarity measure (FESM) branch and overall evaluation network. The Siamese FESM branch extracts the spatial and spectral deep features and calculates the similarity of the corresponding pair of deep features to obtain the spatial and spectral feature parameters, and then, the overall evaluation network realizes the overall quality assessment. Moreover, we propose to quantify both the overall precision of all the training samples and the variations among different fusion methods in a batch, thereby enhancing the network’s accuracy and robustness. The proposed method was trained and tested on a large subjective evaluation dataset comprising 13,620 fused images. The experimental results suggested the effectiveness and the competitive performance.