18 December 2019 in IEEE Transactions on Pattern Analysis and Machine Intelligence
The majority of existing models of person re-identification(re-id) that matches pedestrian images across different camera views are largely dependent on color appearance. However, this can be an issue if that person changes his/her clothes, causing most existing methods to fail. We call person re-id under clothing change the "cross-clothes person re-id". Particularly, as a first attempt at solving this problem using visible light images, we assume a person only changes his/her clothes moderately when the weather doesn't change substantially within a short period so that the shape of a person would not change significantly. We perform cross-clothes person re-id based on a contour sketch of person image to take advantage of the shape of human body instead of color information. To select more discriminative curve patterns on a body contour sketch, we introduce a learning-based spatial polar transformation(SPT) layer in convolutional neural network(CNN) to transform contour sketch images into a polar coordinate space. An angle-specific extractor(ASE) is applied in the CNN to extract more fine-grained discriminant angle-specific features. Finally, we develop a multistream network for aggregating multi-granularity features. We also contribute a cross-clothes dataset consisting of 33698 images from 221 identities. Our experiments demonstrate the effectiveness of our proposed method.
24 September 2021 in IEEE Transactions on Multimedia
Although person re-identification (person re-id) has advanced substantially in recent years, most methods are based on the assumption that the identities would not change clothes. This assumption might not hold in practice considering criminals who intentionally change clothes. In this work, we attempt to solve person re-id under moderate clothing change. Since the human body shape is considered as relatively more invariant under moderate clothing changes, we propose to learn a reliable shape-aware feature representation by mutually learning both colorful images and contour images. Instead of directly extracting shape features from contour images, we utilize contour feature learning as regularization and excavate more effective shape-aware feature representations from colorful images. We propose a multi-scale appearance and contour deep infomax (MAC-DIM) to maximize mutual information between colorful appearance features and contour shape features, and in this way, the extracted appearance features are constrained to be shape-aware in terms of both low-level visual properties and high-level semantics. To better model the long-range human body shape and explicitly capture contour segment relations, we introduce hierarchical graph modeling as aggregation headers, propagating structural context through graph convolutional networks (GCNs). The extensive results on benchmarks under clothing changes demonstrate the effectiveness of our shape-aware feature learning scheme.