6 December 2019 in Multimedia Tools and Applications
2 December 2020 in IEEE Transactions on Multimedia
Person retrieval largely relies on the appearance features of pedestrians. This task is rather more difficult in surveillance videos due to the limitations of extracting robust appearance features brought by the cross-view and cross-camera data with lower image resolution, motion blur, occlusion and other kinds of image degradation. To build up a more reliable person retrieval system, recent works introduced appearance attribute models to describe and distinguish different persons with high-level semantic concepts. Despite the progress of previous works, the value of utilizing appearance attributes is still under-explored. On one hand, existing methods lack for concise and precise attribute representations that are specific for each attribute category and, in the meantime, are able to filter noisy information in irrelevant spatial locations and useless patterns. On the other hand, correlation and reasoning between different attributes are neglected, which could generate more useful information and add more robustness to the retrieval system. In this paper, we propose an Attribute Mining and Reasoning (AMR) framework which is capable to handle the issues in question. The AMR makes better use of appearance attributes with two main components. First, the AMR disentangles the representations of different attributes by localizing their spatial positions and identifying their effective patterns in a weakly supervised manner. To achieve more reliable localization, we propose the Attribute Localization Ensemble (ALE) module that is consisted of multiple localization heads and a voting mechanism. Second, we introduce the Attribute Reasoning (AR) module to correlate different attributes together with the global appearance features and discover their latent relations to generate more comprehensive descriptions of pedestrians. Extensive experiments on DukeMTMC-ReID and Market-1501 datasets demonstrate the effectiveness of the proposed AMR framework as well as its superiority over the existing state-of-the-art methods. The AMR model also shows great generalization ability on the unseen CUHK03 dataset when it is only trained on Market-1501 dataset.
1 September 2019 in IEEE Transactions on Image Processing
In this paper, we propose a design scheme for deep learning networks in the face parsing task with promising accuracy and real-time inference speed. By analyzing the differences between the general image parsing task and face parsing task, we first revisit the structure of traditional FCN and make improvements to adapt to the unique properties of the face parsing task. Especially, the concept of Normalized Receptive Field is proposed to give more insights on designing the network. Then, a novel loss function called Statistical Contextual Loss is introduced, which integrates richer contextual information and regularizes features during training. For further model acceleration, we propose a semi-supervised distillation scheme that effectively transfers the learned knowledge to a lighter network. Extensive experiments on LFW and Helen dataset demonstrate the significant superiority of the new design scheme on both efficacy and efficiency.
10 September 2020 in IEEE Transactions on Multimedia
Person retrieval aims at effectively matching the pedestrian images over an extensive database given a specified identity. As extracting effective features is crucial in a high-performance retrieval system, recent significant progress was achieved by part-based models that have constructed robust local representations on top of vertically striped part features. However, this kind of models use predefined partitioning strategies, making the number and size of each partition identical even when input images vary a lot. This unchangeable setting usually leads to less flexibility and robustness in capturing visual variance. The primary reason for such a negative effect is that a fixed partitioning strategy is unable to deal with (a) the significant variance from pose, illumination and viewpoint which is common in a pedestrian image dataset, and (b) also the inference error and misalignment of human bodies introduced by the prepositive pedestrian detection module or human pose estimation module. In this paper, we tackle this problem via introducing the novel Adaptive Partition Network (APN). The APN utilizes deep reinforcement learning and applies an agent to generate optimal partitioning strategies dynamically for different input images. The agent inside the APN is optimized with the policy gradient algorithm and maximizes the reward of choosing the best partition setting. By leveraging the supervision cues from the objective partitioning strategies that are generated on a set of held-out training images, the agent is trained jointly with other parts of APN, which ensures the APN's robustness and generalization ability. Extensive experimental results on multiple datasets, including CUHK03, DukeMTMC and Market-1501, demonstrate the superiority of APN over the state-of-the-art models.
2 February 2024 in IEEE Transactions on Biometrics, Behavior, and Identity Science
Occluded person re-identification is a challenging task due to various noise introduced by occlusion. Previous methods utilize body detectors to exploit more clues which are overdependent on accuracy of detection results. In this paper, we propose a model named Cascade Transformer Reasoning Embedded by Uncertainty Network (CTU) which does not require external information. Self-attention of the transformer models long-range dependency to capture difference between pixels, which helps the model focus on discriminative information of human bodies. However, noise such as occlusion will bring a high level of uncertainty to feature learning and makes self-attention learn undesirable dependency. We invent a novel structure named Uncertainty Embedded Transformer (UT) Layer to involve uncertainty in computing attention weights of self-attention. Introducing uncertainty mechanism helps the network better evaluate the dependency between pixels and focus more on human bodies. Additionally, our proposed transformer layer generates an attention mask through Cascade Attention Module (CA) to guide the next layer to focus more on key areas of the feature map, decomposing feature learning into cascade stages. Extensive experiments over challenging datasets Occluded-DukeMTMC, P-DukeMTMC, etc., verify the effectiveness of our method.
17 July 2023 in IEEE Transactions on Computational Social Systems
Face recognition (FR) models can be easily fooled by adversarial examples, which are crafted by adding imperceptible perturbations on benign face images. The existence of adversarial face examples poses a great threat to the security of society. To build a more sustainable digital nation, in this article, we improve the transferability of adversarial face examples to expose more blind spots of the existing FR models. Though generating hard samples has shown its effectiveness in improving the generalization of models in training tasks, the effectiveness of using this idea to improve the transferability of adversarial face examples remains unexplored. To this end, based on the property of hard samples and the symmetry between training tasks and adversarial attack tasks, we propose the concept of hard models, which have similar effects as hard samples for adversarial attack tasks. Using the concept of hard models, we propose a novel attack method called beneficial perturbation feature augmentation attack (BPFA), which reduces the overfitting of adversarial examples to surrogate FR models by constantly generating new hard models to craft the adversarial examples. Specifically, in the backpropagation, BPFA records the gradients on preselected feature maps and uses the gradient on the input image to craft the adversarial example. In the next forward propagation, BPFA leverages the recorded gradients to add beneficial perturbations on their corresponding feature maps to increase the loss. Extensive experiments demonstrate that BPFA can significantly boost the transferability of adversarial attacks on FR.