7 October 2021 in IEEE Transactions on Image Processing
Pedestrian detection is a challenging and hot research topic in the field of computer vision, especially for the crowded scenes where occlusion happens frequently. In this paper, we propose a novel AutoPedestrian scheme that automatically augments the pedestrian data and searches for suitable loss functions, aiming for better performance of pedestrian detection especially in crowded scenes. To our best knowledge, it is the first work to automatically search the optimal policy of data augmentation and loss function jointly for the pedestrian detection. To achieve the goal of searching the optimal augmentation scheme and loss function jointly, we first formulate the data augmentation policy and loss function as probability distributions based on different hyper-parameters. Then, we apply a double-loop scheme with importance-sampling to solve the optimization problem of data augmentation and loss function types efficiently. Comprehensive experiments on two popular benchmarks of CrowdHuman and CityPersons show the effectiveness of our proposed method. In particular, we achieve 40.58% in MR on CrowdHuman datasets and 11.3% in MR on CityPersons reasonable subset, yielding new state-of-the-art results on these two datasets.
19 May 2023 in IEEE Transactions on Image Processing
Pedestrian detection is still a challenging task for computer vision, especially in crowded scenes where the overlaps between pedestrians tend to be large. The non-maximum suppression (NMS) plays an important role in removing the redundant false positive detection proposals while retaining the true positive detection proposals. However, the highly overlapped results may be suppressed if the threshold of NMS is lower. Meanwhile, a higher threshold of NMS will introduce a larger number of false positive results. To solve this problem, we propose an optimal threshold prediction (OTP) based NMS method that predicts a suitable threshold of NMS for each human instance. First, a visibility estimation module is designed to obtain the visibility ratio. Then, we propose a threshold prediction subnet to determine the optimal threshold of NMS automatically according to the visibility ratio and classification score. Finally, we re-formulate the objective function of the subnet and utilize the reward-guided gradient estimation algorithm to update the subnet. Comprehensive experiments on CrowdHuman and CityPersons show the superior performance of the proposed method in pedestrian detection, especially in crowded scenes.
10 November 2023 in IEEE Transactions on Multimedia
Occluded person re-identification (Re-ID) is a challenging task, as various object-to-person (OTP) and person-to-person (PTP) occlusion scenarios cause diverse occlusion interference and target person feature loss problems in person matching. Most existing methods, which utilize auxiliary models to evaluate the unoccluded person parts for occlusion feature elimination, are inefficient and cannot handle the PTP occlusion scenarios and person feature loss problems. To solve these issues, we propose a novel Occlusion-Aware Feature Recover (OAFR) model. OAFR simulates diverse occlusions to facilitate the model perceiving OTP, PTP occlusions and recovers occluded query features with unoccluded retrieved gallery features. Concretely, the Prior Knowledge-based Occlusion Simulation method is firstly introduced to synthesize OTP, PTP occlusions and corresponding occlusion labels, empowering model target person perception and occlusion-aware capability through self-supervised learning. Afterward, the feature recovery module reconstructs occluded query features with corresponding unoccluded local features of the top- $K$ retrieved images by the visibility weighted average scheme, thus recovering the occluded query features to maintain more comprehensive features for better retrieval. Extensive experiments demonstrate that the proposed OAFR achieves superior performance to the state-of-the-art for both holistic and occluded Re-ID. Especially for Occluded-DukeMTMC dataset, OAFR outperforms the state-of-the-art by 6.0% for Rank-1 accuracy and 2.2% for mAP score. The source codes are available at https://github.com/yuanbianGit/OAFR .
28 November 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Pedestrian detection currently suffers from two issues in crowded scenes: occlusion and dense boundary prediction, making it still challenging in complex real-world scenarios. In recent years, Convolutional Neural Networks (CNN) and Vision Transformers (ViT) have shown their superiorities in addressing these issues, where ViTs capture global feature dependency to infer occlusion parts and CNNs make accurate dense predictions by local detailed features. Nevertheless, limited by the narrow receptive field, CNNs fail to infer occlusion parts, while ViTs tend to ignore local features that are vital to distinguish different pedestrians in the crowd. Therefore, it is essential to combine the advantages of CNN and ViT for pedestrian detection. However, manually designing a specific CNN and ViT hybrid network requires enormous time and resources for trial and error. To address this issue, we propose the first Neural Architecture Search (NAS) framework specifically designed for pedestrian detection named NAS-PED, which automatically designs an appropriate CNNs and ViTs hybrid backbone for the crowded pedestrian detection task. Specifically, we formulate transformers and convolutions with various kernel sizes in the same format, which provides an unconstrained space for diverse hybrid network search. Furthermore, to search for a suitable backbone, we propose an information bottleneck based NAS objective function, which treats the process of NAS as an information extraction process, preserving relevant information and suppressing redundant information from the dense pedestrians in crowd scenes Extensive experiments on CrowdHuman, CityPersons and EuroCity Persons datasets demonstrate the effectiveness of the proposed method. Our NAS-PED obtains absolute gains of 4.0% MR $^{-2}$ and 1.9% AP over the state-of-the-art (SOTA) pedestrian detection framework on CrowdHuman datasets. For the CityPersons and EuroCity Persons datasets, the searched backbone achieves stable improvement across all three subsets, outperforming some large language-image pre-trained models. Code will be released after acceptance.
25 May 2022 in IEEE Transactions on Instrumentation and Measurement
Pedestrian detection is a challenging task in practical scenarios. Data augmentation is an effective way to deal with this problem, but designing a suitable data augmentation policy requires a great deal of manual experience and multiple adjustments of parameters. To address this problem, we propose an impartial differentiable automatic data augmentation (IDADA) that automatically searches a suitable augmentation policy for pedestrian detection in a more practical scenario. First, an impartiality loss is proposed to solve the bias of the operations probabilities distribution, so as to improve the impartiality for each operation. Second, the impartiality loss and the original validation loss of the detection model are combined as a target function to update the augmentation policy parameters. Since the target function is not differentiable, the finite difference approximation is applied to provide accurate gradient estimation for the augmentation policy optimization process. Comprehensive experiments performed on two typical datasets of CrowdHuman and CityPersons verify the effectiveness of the proposed approach. Specifically, our approach achieves 40.99% in miss rate (MR) on CrowdHuman, yielding the new state-of-the-art result on this dataset.
28 November 2023 in IEEE Transactions on Instrumentation and Measurement
Data augmentation (DA) is a commonly used method to alleviate the problem of detecting occluded pedestrians in crowded scenes. Recently, several dataset-level automatic data augmentation methods have been proposed to search for a set of general DA policies for the entire dataset, which saves a lot of time compared to manually designing DA policies. However, due to the huge differences between each image in pedestrian detection datasets, existing dataset-level augmentation methods cannot automatically adjust DA policies according to the differences between them, which will lead to outlier data and degradation of model performance. Therefore, considering the differences between each image, we propose an image-level automatic data augmentation method that aims to find an optimal DA policy for each image in the dataset according to their respective characteristics. Specifically, we first reformulate the image-level automatic data augmentation method by constructing a search space based on Categorical distribution, within which we specify the probability of operations being sampled according to their respective effectiveness, so that useful operations can be effectively preserved and useless operations can be suppressed. Subsequently, we design an encoding method to recode the index of images and policies, and use the encoded index to closely associate them to achieve a stable matching relationship between images and policies. Finally, a search framework with Bayesian optimization is developed for efficient policy mining. Comprehensive experiments on CrowdHuman and CityPersons datasets show that compared with the commonly used automatic data augmentation method for pedestrian detection, AutoPedestrian, our method takes only 1/14 of the search time, but achieves better detection accuracy. Specifically, we achieve 10.2% MR -2 on CityPersons reasonable subset and 36.8% MR -2 on CrowdHuman dataset, outperforming state-of-the-art methods on CrowdHuman dataset.