Visible-Infrared person reidentification (VI-ReID) is a challenging matching problem due to large modality variations between visible and infrared images. Existing approaches usually bridge the modality gap with only feature-level constraints, ignoring pixel-level variations. Some methods employ a generative adversarial network (GAN) to generate style-consistent images, but it destroys the structure information and incurs a considerable level of noise. In this article, we explicitly consider these challenges and formulate a novel spectrum-aware feature augmentation network named SFANet for cross-modality matching problem. Specifically, we put forward to employ grayscale-spectrum images to fully replace RGB images for feature learning. Learning with the grayscale-spectrum images, our model can apparently reduce modality discrepancy and detect inner structure relations across the different modalities, making it robust to color variations. At feature level, we improve the conventional two-stream network by balancing the number of specific and sharable convolutional blocks, which preserve the spatial structure information of features. Additionally, a bidirectional tri-constrained top-push ranking loss (BTTR) is embedded in the proposed network to improve the discriminability, which efficiently further boosts the matching accuracy. Meanwhile, we further introduce an effective dual-linear with batch normalization identification (ID) embedding method to model the identity-specific information and assist BTTR loss in magnitude stabilizing. On SYSU-MM01 and RegDB datasets, we conducted extensively experiments to demonstrate that our proposed framework contributes indispensably and achieves a very competitive VI-ReID performance.
4 January 2023 in IEEE Journal of Selected Topics in Signal Processing
Visible-infrared person re-identification (VI-ReID) is a challenging and essential task that aims to retrieve a set of person images over visible and infrared camera views. In order to mitigate the impact of large modality discrepancy existing in heterogeneous images, previous methods attempt to apply generative adversarial network (GAN) to generate the modality-consisitent data. However, due to severe color variations between the visible domain and infrared domain, the generated fake cross-modality samples often fail to possess good qualities to fill the modality gap between synthesized scenarios and target real ones, which leads to sub-optimal feature representations. In this work, we address the cross-modality matching problem with Aligned Grayscale Modality (AGM), an unified dark-line spectrum that reformulates visible-infrared dual-mode learning as a gray-gray single-mode learning problem. Specifically, we generate the grayscale modality from the homogeneous visible images. Then, we train a style tranfer model to transfer infrared images into homogeneous grayscale images. In this way, the modality discrepancy is remarkably reduced in the image space. In order to reduce the remaining appearance discrepancy, we further introduce a multi-granularity feature extraction network to conduct feature-level alignment. Rather than relying on the global information, we propose to exploit local (head-shoulder) features to assist person Re-ID, which complements each other to form a stronger feature descriptor. Comprehensive experiments implemented on the mainstream evaluation datasets include SYSU-MM01 and RegDB indicate that our method can remarkably enhance cross-modality retrieval performance against the state of the art methods.
18 February 2023 in Pattern Analysis and Applications
Visible-thermal person re-identification (VT-ReID) is an emerging cross-modality matching problem, which aims to identify the same person across the daytime visible modality and nighttime thermal modality in the Internet of Things. Existing cutting-edge approaches consistently attempts to exploit image generation technique to generate cross-modality images or design various feature-level constraints to align feature distribution of heterogeneous data. However, color variations originating from the different imaging processes of spectrum cameras remains unsolved, which leads to sub-optimal feature representations. In this paper, we present a simple but very effective data augmentation method named Random Gray for the cross-modality matching task. Given a training sample, Random Gray randomly selects a rectangular region and translates it to grayscale. In this process, training images with fusing various levels of visible and grayscale information are generated, thereby reducing the risk of overfitting and making the model robust to color variations. Besides, we introduce a novel pooling method called softpooling to retain more information in the reduced activation maps. With softpooling layer, the network can learn more discriminative person features and further boost its retrieval performance. We conduct extensive experiments on publicly available cross-modality Re-ID datasets (SYSU-MM01 and RegDB) to demonstrate the effectiveness of our proposed method. Experimental results show that Random Gray and softpooling strategies yield siginificant accurancy improvement, and they can be utilized as training tricks for further VT-ReID research.