Hailin Shi

Hailin Shi

Auto-generated profile, based on publicly available information

Hailin Shi

NIO, Beijing, China

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 January 2022 in ACM Computing Surveys
Verified
Journal Article
Open access
HDHang Du
Hang Du
HSHailin Shi
Hailin Shi
DZDan Zeng
Dan Zeng
XZXiao-Ping Zhang
Xiao-Ping Zhang
TMTao Mei
Tao Mei
Face recognition (FR) is one of the most popular and long-standing topics in computer vision. With the recent development of deep learning techniques and large-scale datasets, deep face recognition has made remarkable progress and has been widely used in many real-world applications. Given a natural image or video frame as input, an end-to-end deep face recognition system outputs the face feature for recognition. To achieve this, a typical end-to-end system is built with three key elements: face detection, face alignment, and face representation. Face detection locates faces in the image or frame. Then, the face alignment is proceeded to calibrate the faces to the canonical view and crop them with a normalized pixel size. Finally, in the stage of face representation, the discriminative features are extracted from the aligned face for recognition. Nowadays, all of the three elements are fulfilled by the technique of deep convolutional neural network. In this survey article, we present a comprehensive review about the recent advance of each element of the end-to-end deep face recognition, since the thriving deep learning techniques have greatly improved their capability of them. To start with, we present an overview of the end-to-end deep face recognition. Then, we review the advance of each element, respectively, covering many aspects such as the to-date algorithm designs, evaluation metrics, datasets, performance comparison, existing challenges, and promising directions for future research. Also, we provide a detailed discussion about the effect of each element on its subsequent elements and the holistic system. Through this survey, we wish to bring contributions in two aspects: first, readers can conveniently identify the methods which are quite strong-baseline style in the subcategory for further exploration; second, one can also employ suitable methods for establishing a state-of-the-art end-to-end face recognition system from scratch.

24 September 2021 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
GTGusi Te
Gusi Te
WHWei Hu
Wei Hu
YLYinglu Liu
Yinglu Liu
HSHailin Shi
Hailin Shi
TMTao Mei
Tao Mei
Face parsing infers a pixel-wise label to each facial component, which has drawn much attention recently. Previous methods have shown their success in face parsing, which however overlook the correlation among facial components. As a matter of fact, the component-wise relationship is a critical clue in discriminating ambiguous pixels in facial area. To address this issue, we propose adaptive graph representation learning and reasoning over facial components, aiming to learn representative vertices that describe each component, exploit the component-wise relationship and thereby produce accurate parsing results against ambiguity. In particular, we devise an adaptive and differentiable graph abstraction method to represent the components on a graph via pixel-to-vertex projection under the initial condition of a predicted parsing map, where pixel features within a certain facial region are aggregated onto a vertex. Further, we explicitly incorporate the image edge as a prior in the model, which helps to discriminate edge and non-edge pixels during the projection, thus leading to refined parsing results along the edges. Then, our model learns and reasons over the relations among components by propagating information across vertices on the graph. Finally, the refined vertex features are projected back to pixel grids for the prediction of the final parsing map. To train our model, we propose a discriminative loss to penalize small distances between vertices in the feature space, which leads to distinct vertices with strong semantics. Experimental results show the superior performance of the proposed model on multiple face parsing datasets, along with the validation on the human parsing task to demonstrate the generalizability of our model.

8 April 2021 in IEEE Signal Processing Letters
Verified
Journal Article
Closed access
HDHang Du
Hang Du
HSHailin Shi
Hailin Shi
YLYinglu Liu
Yinglu Liu
DZDan Zeng
Dan Zeng
TMTao Mei
Tao Mei
Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition, which aims to match a pair of face images captured from two different modalities. Existing deep learning based methods have made remarkable progress in NIR-VIS face recognition, while it encounters certain newly-emerged difficulties during the pandemic of COVID-19, since people are supposed to wear facial masks to cut off the spread of the virus. We define this task as NIR-VIS masked face recognition, and find it problematic with the masked face in the NIR probe image. First, the lack of masked face data is a challenging issue for the network training. Second, most of the facial parts (cheeks, mouth, nose etc.) are fully occluded by the mask, which leads to a large amount of loss of information. Third, the domain gap still exists in the remaining facial parts. In such scenario, the existing methods suffer from significant performance degradation caused by the above issues. In this paper, we aim to address the challenge of NIR-VIS masked face recognition from the perspectives of training data and training method. Specifically, we propose a novel heterogeneous training method to maximize the mutual information shared by the face representation of two domains with the help of semi-siamese networks. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image. Resorting to these practices, our solution provides the domain-invariant face representation which is also robust to the mask occlusion. Extensive experiments on three NIR-VIS face datasets demonstrate the effectiveness and cross-dataset-generalization capacity of our method.

12 July 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
WLWu Liu
Wu Liu
HSHailin Shi
Hailin Shi
YWYunchao Wei
Yunchao Wei
DZDan Zeng
Dan Zeng
NSNicu Sebe
Nicu Sebe
JLJiebo Luo
Jiebo Luo

11 January 2024 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
ZGZheng Gong
Zheng Gong
Xuezhi YangXuezhi Yang
Xuezhi Yang
RSRencheng Song
Rencheng Song
XHXuesong Han
Xuesong Han
CRChong Ren
Chong Ren
WLWei Li
Wei Li
Remote photoplethysmography (rPPG) enables heart rate (HR) measurement under stable illumination and low noise conditions. However, challenges arise due to rapid lighting changes, significant head movements, and vehicle vibrations during driving, impacting the recovery of clear rPPG signals and rendering denoising techniques alone inadequate in entirely eliminating noise interference. To address these challenges, we introduce an innovative approach, “quality-guided spectrum peak screening” (QSPS), for monitoring the driver’s HR during driving. First, we developed a signal evaluation framework for assessing signal quality across multiple facial regions following wavelet filtering. Subsequently, by leveraging quality information, a spectral peak screening algorithm is applied to signals from multiple regions, mitigating the impact of residual noise on rPPG. The precise HR is determined by integrating quality scores and short-term stability from various facial regions. Our method is employed in a commercially available driver monitoring system (DMS) equipped with a monochrome camera. Test results show QSPS’s robustness compared with established near-infrared (NIR)-based HR detection methods. In driving scenarios, QSPS achieves a mean absolute error (MAE) of 4.32 bpm, a root mean square error (RMSE) of 6.15 bpm, and a percentage of time with HR estimation errors smaller than 6 bpm of 68.8%. Furthermore, QSPS exhibits excellent performance during extended 10-min HR monitoring, with a Pearson correlation coefficient of 0.943 in the night test and 0.906 in the day test.

12 July 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
YTYichun Tai
Yichun Tai
HSHailin Shi
Hailin Shi
DZDan Zeng
Dan Zeng
HDHang Du
Hang Du
YHYibo Hu
Yibo Hu
TMTao Mei
Tao Mei
With the recent development of deep convolutional neural networks and large-scale datasets, deep face recognition has made remarkable progress and been widely used in various applications. However, unlike the existing public face datasets, in many real-world scenarios of face recognition, the depth of the training dataset is shallow, which means that only two face images are available for each ID. With the non-uniform increase of samples, such issue is converted to a more general case, known as long-tail face learning, which suffers from data imbalance and intra-class diversity dearth simultaneously. These adverse conditions damage the training and result in the decline of model performance. Based on Semi-Siamese Training, we introduce an advanced solution, namedMulti-Agent Semi-Siamese Training(MASST), to address these problems. MASST includes a probe network and multiple gallery agents—the former aims to encode the probe features, and the latter constitutes a stack of networks that encode the prototypes (gallery features). For each training iteration, the gallery network, which is sequentially rotated from the stack, and the probe network form a pair of Semi-Siamese networks. We give the theoretical and empirical analysis that, given the long-tail (or shallow) data and training loss, MASST smooths the loss landscape and satisfies the Lipschitz continuity with the help of multiple agents and the updating gallery queue. The proposed method is out of extra-dependency, and thus can be easily integrated with the existing loss functions and network architectures. It is worth noting that although multiple gallery agents are employed for training, only the probe network is needed for inference, without increasing the inference cost. Extensive experiments and comparisons demonstrate the advantages of MASST for long-tail and shallow face learning.