Visible-infrared person re-identification (VI Re-ID) is designed to match person images of the same identity from visible and infrared cameras. Transformer structures have been successfully applied in the field of VI Re-ID. However, previous Transformer-based methods were mainly designed to capture global content information in a single modality, and could not simultaneously perceive semantic information between two modalities from a global perspective. To solve this problem, we propose a novel framework named the cross-modality interaction Transformer (CMIT). It has strong abilities in modeling spatial and sequential features that can capture dependencies between long-range features, and explicitly improves the discriminativeness of features by exchanging information across modalities, thus contributing to obtaining modality-invariant representations. Specifically, CMIT utilizes a cross-modality attention mechanism to enrich the feature representations of each patch token by interacting with the patch tokens of the other modality, and aggregates local features of the CNN structure and global information of the Transformer structure to mine feature saliency representation. Furthermore, the modality-discriminative (MD) loss function is proposed to learn potential consistency between modalities to encourage intra-modality compactness within class and inter-modality separation between classes. Extensive experiments on two benchmarks demonstrate that our approach outperforms state-of-the-art methods.
Video-based visible-infrared person re-identification (VVI-ReID) aims to match the identity of a person captured in video sequences from both visible and infrared cameras. The VVI-ReID task requires considering both the spatial relationship between body parts within each frame and the temporal change of appearance between successive frames. Existing VVI Re-ID methods employ Convolutional Neural Networks to extract local spatial features and Long Short-Term Memory to form temporal associations. However, these methods can not effectively capture the global spatial feature and the long-range temporal dependencies in ultra-long sequences. In this paper, we propose a Cross-modality Spatial-temporal Transformer (CST) including a Cross-frame Tube Transformer Module (CTTM) and a Multi-frame Transformer Fusion Module (MTFM) to address these challenges. Firstly, CTTM tokenizes a video clip into multiple 3D tubes, each encapsulating local spatial-temporal information of pedestrians, and then obtains global spatial-temporal representations by establishing the relationship between tubes. Secondly, we design MTFM to exchange information between multiple frames using message tokens, thus modeling the long-range temporal dependencies of features of pedestrians. In addition, to prevent the potential representation collapse caused by triplet-based loss functions, we propose a diversity-consistency (DC) loss function to preserve the diversity and consistency of cross-modality feature representations by imposing variance, invariance, and covariance constraints in feature representations. Extensive benchmark experiments demonstrate that our approach outperforms the state-of-the-art methods with large margins.
Occlusion is a key challenge in real-world person reidentification (ReID), as partial occlusions often remove critical identity cues. Most existing methods focus on feature extraction for visible regions while ignoring the holistic contextual information of pedestrians, leading to incomplete feature representations. This issue is particularly severe in visible-infrared ReID (VI-ReID), where spatial inconsistencies of occluded regions and modality-specific feature degradation jointly exacerbate feature misalignment, but has received little attention in previous studies. Therefore, we propose an Occlusion-Aware Cross-Modality Completion Network (OCCNet) to explicitly complete missing identity cues, which integrates three complementary components. The Graph-guided Feature Completion Transformer (GFCT) embeds a dynamically constructed semantic graph into self-attention to complete occluded tokens within each modality, while the Semantic-Aware Region Completion Module (SRCM) aggregates tokens into coherent regions and adaptively transfers complementary cues across modalities. To ensure semantic coherence, a Multi-Distribution Consistency Alignment (MCA) loss is employed to regularize token- and region-level completions toward a unified distribution. By integrating these components, OCCNet recovers missing identity information at multiple levels and mitigates modality gaps caused by occlusion. To evaluate its effectiveness under realistic occlusion scenarios, we first construct two realistic occluded benchmarks, Occluded-SYSU-MM01 and Occluded-RegDB, on which extensive experiments demonstrate that our OCCNet outperforms state-of-the-art methods.
Mesh adaptation is crucial in numerical simulation, providing optimal resource allocation for accurately capturing physical phenomena. However, when applied to Computational Fluid Dynamics (CFD) problems with complex multi-scale properties, existing adaptation methods face huge challenges due to the high computational cost of solving auxiliary partial differential equations (PDEs) and the difficulty in aligning the flow features with mesh geometric features. In this work, an end-to-end data-driven mesh adaptation framework, Flow2Mesh, is proposed to address these challenges by adopting a hybrid modeling strategy to construct the mapping from pixelated flow-fields to graph-based meshes. It achieves a rapid and accurate one-step mesh adaptation via a perceptual feature network (PFN) and a mesh movement network (MMN). PFN extracts the global perceptual features from flow-fields to enhance flow feature representation and mesh resolution independence. In MMN, these features are utilized to deform the initial mesh to a topology-invariant adaptive mesh by a proposed physically driven mesh convolutional network. It considers the inherent mesh geometric information for efficient node feature aggregation and alignment of mesh density with a flow-field structure. To generate high-quality adaptive meshes, various mesh-related losses are designed to regularize the mesh movement and alleviate the mesh tangling. Experiments in CFD scenarios demonstrate the generalization of our model to different design parameters and mesh configurations. It takes three orders of magnitude less time to generate similar meshes than the PDE-based method. The results exhibit the potential of Flow2Mesh to be a flexible and reliable tool for rapid mesh adaptation in scientific and industrial fields.