25 April 2025 in IEEE Transactions on Geoscience and Remote Sensing
Most existing visual saliency analysis methods are meticulously crafted for natural RGB images, which typically lean upon foundational spatial visual cues, such as contrast, structure and texture in scenes. However, these methods exhibit limitations in spectral saliency analysis due to the existence of numerous materials that may manifest identical RGB values while harboring disparate spectral characteristics. Consequently, it is an exigency to introduce saliency method specially for hyperspectral images (HSIs). In this paper, we establish a set of hyperspectral saliency principles that incorporate both spectral and spatial attributes, and accordingly present a novel self-supervised saliency analysis (S3A) framework for HSIs. Note that our S3A is designed as a discriminant architecture composed of three interconnected components. More specifically, an unsupervised HSI representive patch sampling (RPS) module is designed to pick up some representative pixels for self-supervised saliency analysis training. Subsequently, we construct a classification-based patch spectral discrimination network (CPSD-Net) to evaluate the HSI saliency. Finally, a patch-to-global spatial diffusion (P2G-SD) module is constructed to diffuse the saliency from few supervised samples to the other HSI pixels. Moreover, to demonstrate the performance of our saliency analysis framework, we apply the HSI saliency to some classic downstream tasks including band selection (BS) and HSI classification. On several popular HSI datasets, the satisfactory quantization results fully verify the rationality and effectiveness of our S3A framework, in terms of entropy value and mean spectral divergence (MSD) of the selected bands in BS task, as well as the accuracy in HSI classification task. The source code is publicly available at https://github.com/Lee-zl/S3A.
5 July 2018 in IEEE Access
Fusing panchromatic (PAN) and multispectral (MS) images, i.e., pansharpening, can obtain a high-resolution multispectral image (HRMS). In this paper, we propose a spectral mapping framework for pansharpening problem based on panchromatic block structure analysis (PBSA). PBSA employs boundary, inner uniqueness and neighborhood comparisons as block structure characteristics to classify PAN image blocks into pureness composition and mixture composition. For the PAN blocks with pureness composition, they can directly copy the spectral information of corresponding MS pixels. For the mixture PAN blocks, assuming that they can linearly represented by some pure PAN blocks, their spectral signals can get via a weighted average from the relative pure PAN blocks. The experiments on real PAN and MS pairs show that the proposed pansharpening method not only conforms to the structure of PAN image but also preserves the spectral information of MS image. The final fused HRMS image shows good performance in visual effect and objective assessment.
12 January 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Sign Language Production (SLP) aims to translate spoken textual languages into sign language sequences, which can significantly bridge the communication gap for deaf and hard-of-hearing individuals. Most previous SLP methods typically rely on skeleton-based data, which hinders their realism and expressive capacity. In this work, we address expressive 3D SLP tasks to generate high-quality 3D holistic sign motions driven by spoken language. However, existing 3D SLP methods struggle to accurately capture spatial relationships within intricate 3D structures and overlook the alignment of semantics at the word level. To overcome these limitations, we propose SignMask, a novel generative masked modeling framework that enhances spatial structure awareness and semantic understanding. We first design a structural holistic sign motion tokenizer that hierarchically learns discrete tokens of body and hand movements. This tokenizer adaptively aggregates 3D SMPL-X pose features corresponding to the same semantic parts and dynamically adjusts the weights between pose features of different semantic parts, enhancing spatial awareness and ensuring semantic consistency. Building on these tokenized representations, we introduce a specialized Sign-M Transformer to learn masked token prediction guided by textual input. Our Sign-M Transformer employs a hierarchical masking strategy, alongside spatio-temporal and cross-modal attention mechanisms, to effectively capture complex spatio-temporal relationships among sign tokens and semantic dependencies between sign and text tokens. During inference, our SignMask model parallelly and iteratively fills up the missing motion tokens starting from full-masked token sequences, therefore achieving high-fidelity and efficient 3D sign avatar generation. Extensive experiments demonstrate the superior performance of our approach compared to existing SLP methods across various lingual sign language datasets in generating high-quality and semantically consistent sign language motions.