18 March 2024 in IEEE Transactions on Circuits and Systems for Video Technology
The aim of end-to-end sign language translation (SLT) is to interpret continuous sign language (SL) video sequences into coherent natural language sentences without any intermediary annotations, i.e., glosses. However, end-to-end SLT suffers several intractable issues: (i) the temporal correspondence constraint loss problem between SL videos and glosses, and (ii) the weakly supervised sequence labeling problem between long SL videos and sentences. To address these issues, we propose an adaptive video representation enhanced Transformer (AVRET), with three extra modules: adaptive masking (AM), local clip self-attention (LCSA) and adaptive fusion (AF). Specifically, we utilize the first AM module to generate a special mask that adaptively drops out temporally important SL video frame representations to enhance the SL video features. Then, we pass the masked video feature to the Transformer encoder consisting of LCSA and masked self-attention to learn clip-level and continuous video-level feature information. Finally, the output feature of encoder is fused with the temporal feature of AM module via the AF module and use the second AM module to generate more robust feature representations. Besides, we add weakly supervised loss terms to constrain these two AM modules. To promote the Chinese SLT research, we further construct CSL-FocusOn, a Chinese continuous SLT dataset, and share its collection method. It involves many common scenarios, and provides SL sentence annotations and multi-cue images of signers. Our experiments on the CSL-FocusOn, PHOENIX14T, and CSL-Daily datasets show that the proposed method achieves the competitive performance on the end-to-end SLT task without using glosses in training. The code is available at https://github.com/LzDddd/AVRET.
20 November 2024 in IEEE Transactions on Instrumentation and Measurement
Low-dose computed tomography (LDCT) reduces radiation exposure but suffers from high noise, impacting image quality and diagnostic accuracy. Supervised learning has helped address this challenge but requires numerous paired datasets of LDCT and normal-dose CT (NDCT) images, which limits their clinical practice. This article proposes a novel self-supervised disentangled convolutional neural network (SDCNN) that can directly reconstruct high-quality CT images from LDCT data without the need for a clean reference. Unlike other methods that treat noise as a uniform entity, SDCNN disentangles LDCT images into noise-free images, signal-dependent noise, and signal-independent noise, aligning with the intrinsic principles of low-dose noise generation. To enhance the purity of disentanglement, we introduce the concept of combination and re-disentanglement to establish a training framework based on SDCNN. Additionally, we design self-supervised loss functions, including novel anisotropic total variation (TV) and distance loss functions, to improve the efficiency of the denoising process. The signal-guided attention (SGA) module effectively captures the relationship between signal-dependent noise and the signal across both spatial and channel dimensions. Experiments on clinical and animal data demonstrate that the proposed method performed better than all competing state-of-the-art (SOTA) self-supervised algorithms in noise and artifact removal. For example, compared to the self-supervised algorithms, SDCNN improves the multiscale structural similarity (MSSIM), peak signal-to-noise ratio (PSNR), and feature similarity (FSIM) by at least 2.26%, 1.20 dB, 1.23%, whereas, the gradient magnitude similarity deviation (GMSD) is reduced by at least 1.11% on the Mayo clinical data. The code is available at https://github.com/YuhangLiu98/SDCNN .
23 October 2025 in Expert Systems with Applications
9 June 2025 in IEEE Sensors Journal
EEG, which uses electrodes as highly sensitive sensors to capture the brain’s electrical activity, is used to study depression. With the development of GNN, topological analysis based on high-density electrodes has become an important research direction in depression detection. However, effective capture of complex spatial relationships between electrodes, particularly long-range dependencies across brain regions, remains challenging. In this work, we propose a method called GNMixer, which enhances the capability of topology representation learning through a local and global information interaction strategy. Specifically, we divide the entire cerebral cortex into multiple subbrain regions, which are considered distinct patches. Inside each patch, we develop a brain patch encoder based on P2GNN to enhance the attention of local features. During the encoding process, we employ a precomputed approach to extract multichannel topological features. Outside the patches, we use the MLP-Mixer to capture interactions between brain regions and explore macroscopic connectivity patterns. In the MODMA dataset, the accuracy of GNMixer reaches 93.12%. Our method promotes the advancement of high-density EEG sensor technology in depression detection.
24 November 2025 in Expert Systems with Applications
1 October 2026 in IEEE Transactions on Medical Imaging
Few-shot Medical Image Segmentation (FS-MIS) has garnered increasing attention for its ability to reduce reliance on large-scale pixel-wise annotations. How ever, most existing methods rely solely on single-level encoder outputs, neglecting the complementary roles of hi erarchical features in contour refinement and regional dis crimination. This limitation often results in boundary ambi guity and local segmentation errors. To this end, we pro pose a Triple-Attention Coordinated Cross-layer Dynamic Network (TriCD-Net) for FSMIS. Specifically, we design a Feature-Enhanced Boundary Refinement (FEBR) module in each encoder layer, where three parallel attention branches explicitly model foreground, background, and boundary regions. The resulting features are passed to a Dense Dual-Contrast Segmentation (DDCS) head to perform segmenta tion, enabling progressive boundary refinement. Consider ing that query images inherently contain valuable structural information, we introduce a mask reconstruction task and design a Cross-layer Dynamic Convolution Fusion (CDCF) module that dynamically generates convolution kernels to adaptively fuse multi-level features for query reconstruc tion. Furthermore, we establish an interaction between the reconstruction and segmentation branches through an Uncertainty-Gated Cross-Attention Injection (UCI) mod ule, which selectively injects reconstruction features into the segmentation branch to enhance structural coher ence and boundary accuracy. Extensive experiments on three public benchmark datasets demonstrate that TriCD-Net consistently achieves state-of-the-art performance. The code is available at https://github.com/qchi-code/TriCD-Net.