12 March 2026 in IEEE Transactions on Circuits and Systems for Video Technology
Crowd counting has long been a crucial topic in the domains of computer vision and video surveillance. In particular, with the widespread use of thermal or depth cameras, RGB-X crowd counting has emerged as a prominent research focus. Although depth or infrared images provide complementary information, the core challenge remains in effectively unifying heterogeneous cross-modality and cross-scale information to form a comprehensive representation of crowd distributions in complex scenes. To address this problem, we propose a novel Mamba-based framework, termed UMS-VMamba-CC for multi-modal (RGB-X) crowd counting. Specifically, we design the cross-modality disentanglement fusion visual Mamba (CMDF-VMamba) that uses self-supervised learning to decompose modality-invariant and modality-specific features in spatial-frequency domains, followed by multi-modal feature aggregation via the gating mechanism. For cross-scale fusion, we design the cross-scale pyramid fusion visual Mamba (CSPF-VMamba), which adopts the bi-directional pyramid structure to incorporate low-level features into high-level representations during downsampling, while upsampling high-level features and aggregating them with low-level features through state space contextual modeling. Comprehensive experiments on multiple mainstream datasets demonstrate that the UMS-VMamba-CC framework achieves competitive performance for RGB-X crowd counting.
24 February 2026 in IEEE Sensors Journal
RGB-thermal (RGB-T) tracking leverages the complementary advantages of visible light and thermal infrared sensors to enhance robustness under varying illumination, occlusion, and background clutter. However, existing fusion methods still suffer from redundant feature representations and insufficient cross-modal interaction, which constrain their adaptability in complex environments. To overcome these challenges, we propose an Adaptive Cross-Modality Interaction (ACMI) framework based on the vision transformer (ViT), which enables dynamic and bidirectional information exchange between red–green–blue (RGB) and thermal infrared( TIR) modalities. Within ACMI, Shallow Feature Enhancement (SFE) submodule adaptively reweights modality-specific shallow features in a data-driven manner to strengthen discriminative representations; the DynamicFusion module jointly models spatial–channel dependencies to enhance informative cross-sensor correlations; SEAdapter introduces residual calibration to adaptively balance modality contributions and mitigate information loss. These modules are embedded throughout the Transformer layers, forming a hierarchical architecture that progressively strengthens cross-modal feature understanding. Extensive experiments demonstrate that ACMI achieves consistent performance improvements across multiple RGB-T tracking benchmarks, particularly in low-resolution and cluttered scenes, while maintaining real-time performance. The results validate ACMI as a robust and adaptive framework for efficient multi-sensor visual tracking.
9 April 2026 in IEEE Transactions on Intelligent Transportation Systems
Road scene semantic segmentation is one of the most crucial tasks in autonomous driving. Unlike general semantic segmentation, traffic applications require fast image processing, making it difficult to satisfy both high-performance and real-time. Moreover, how to efficiently fuse multi-scale feature representations for a comprehensive understanding of the road scene remains challenging. To address these issues, we propose a lightweight U-shape deep learning framework for high-performance and real-time road scene semantic segmentation in this paper. Specifically, we design the lightweight mixed convolutional block (LMCB), which combines depthwise separable convolution and asymmetric dilated convolution, with the purpose of reducing computational cost and expanding the receptive field for road scene feature extraction. Then, efficient multi-scale feature fusion Transformer (EMFFTrans) is proposed to bridge the encoder-decoder interaction, leveraging the separable cross-channel attention (SCCA) mechanism to capture the inherent dependencies across multi-scale feature maps, thereby facilitating comprehensive representation for road scenes. Experimental results show that our proposed framework, running on a single RTX 3090 GPU card, strikes a satisfactory balance between accuracy and efficiency. Specifically, the LMCB-EMFFTrans-Slim model achieves 73.7% mIOU at 166 FPS on the CamVid dataset and 76.3% mIOU at 113 FPS on the Cityscapes dataset with fewer than 1 million learnable parameters. The code will be soon available at: https://github.com/WH-IPUSLab/EMFFTrans