Han Xue

Han Xue

Auto-generated profile, based on publicly available information

Han Xue

Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 February 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
HXHan Xue
Han Xue
JLJun Ling
Jun Ling
ATAnni Tang
Anni Tang
LSLi Song
Li Song
RXRong Xie
Rong Xie
WZWenjun Zhang
Wenjun Zhang
Face reenactment aims to generate an animation of a source face using the poses and expressions from a target face. Although recent methods have made remarkable progress by exploiting generative adversarial networks, they are limited in generating high-fidelity and identity-preserving results due to the inappropriate driving information and insufficiently effective animating strategies. In this work, we propose a novel face reenactment framework that achieves both high-fidelity generation and identity preservation. Instead of sparse face representations (e.g., facial landmarks and keypoints), we utilize the Projected Normalized Coordinate Code (PNCC) to better preserve facial details. We propose to reconstruct the PNCC with the source identity parameters and the target pose and expression parameters estimated by 3D face reconstruction to factor out the target identity. By adopting the reconstructed representation as the driving information, we address the problem of identity mismatch. To effectively utilize the driving information, we establish the correspondence between the reconstructed representation and the source representation based on the features extracted by an encoder network. This identity-matched correspondence is then utilized to animate the source face using a novel feature transformation strategy. The generator network is further enhanced by the proposed geometry-aware skip connection. Once trained, our model can be applied to previously unseen faces without further training or fine-tuning. Through extensive experiments, we demonstrate the effectiveness of our method in face reenactment and show that our model outperforms state-of-the-art approaches both qualitatively and quantitatively. Additionally, the proposed PNCC reconstruction module can be easily inserted into other methods and improve their performance in cross-identity face reenactment.

1 February 2022 in Journal of Physics: Conference Series
Verified
Conference Paper
Open access
SMSongzhe Ma
Songzhe Ma
HLHuimin Lu
Huimin Lu
YWYifan Wang
Yifan Wang
HXHan Xue
Han Xue
With the continuous development of network, the increasing popularity of embedded technology, and the continuous improvement of mobile terminal computing ability, how to reduce the size of the model without affecting the accuracy as much as possible, and how to deploy the algorithm in embedded devices has become the current research hotspot. In this paper, we propose a new target detection algorithm called YOLOX-Mobile. Firstly, in order to further reduce the power consumption of embedded devices while maintaining the performance, we replaced the SiLU activation function in the original YOLOX with Mish activation function. Because SiLU activation functions occupy a certain amount of computing and storage resources, and the cost of calculating the Sigmoid type functions on mobile devices is much higher. Therefore, we use Mish functions that are smoother, non-monotonic, unbounded upper and lower bounds are used as activation functions to better meet the low power requirements of embedded devices. Secondly, we used Focal Loss as the Loss function of obj_output to achieve the balance of positive and negative samples, as well as hard-to-classify and easy-to-classify samples. Thirdly, we introduce the Involution operator and use it as the convolution kernel of 3×3. We validated the proposed algorithm on public dataset VOC2012 against the current mainstream YOLOX, YOLOX-M, and YOLOX-S algorithms. The mAP of our proposed algorithm is 78.22% and the detection speed is 55.26 FPS. Compared with the original YOlOX, YOLOX-M and YOLOX-S algorithms, the average FPS is improved by 1.99% and 4.13 FPS, and the average Params Size is reduced by 28.80%. Experimental results show that our proposed algorithm improves the accuracy and speed of detection on top of greatly reducing the network parameters and computation, making it a more suitable target detection algorithm for mobile devices.

25 November 2024 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
JLJun Ling
Jun Ling
HXHan Xue
Han Xue
ATAnni Tang
Anni Tang
RXRong Xie
Rong Xie
LSLi Song
Li Song
Unsupervised face reenactment aims to animate a source image to imitate the motions of a target image while retaining the source portrait’s attributes like facial geometry, identity, hair texture, and background. While prior methods can extract the motion from the target image via compact representations (e.g., keypoints or latent motion bases [ 50 ]), they are not robust in predicting motions that are disentangled with portrait attributes, thus failing to preserve portrait attributes in the cross-subject reenactment. In this work, we propose an effective and cost-efficient face reenactment approach to address this issue. Our approach is highlighted by two major strengths. First, based on the theory of latent motion bases, we disentangle the full-head motion into two parts: the transferable motion and preservable motion and then compose the full motion representation using latent motions from the source image and the target image. Second, to optimize and learn disentangled motions, we introduce an efficient training framework, which features two training strategies: (1) a mixture training strategy that encompasses self-reenactment training and cross-subject training for better motion disentanglement and (2) a multi-path training strategy that improves the visual consistency of portrait attributes. Extensive experiments on widely used benchmarks demonstrate that our method exhibits a remarkable generalization ability compared to state-of-the-art baselines. Project and demos are available at https://junleen.github.io/projects/vicoface .

1 January 2023 in International Journal of Computer Applications in Technology
Verified
Journal Article
Closed access
ZLZun Liu
Zun Liu
XHXue Han
Xue Han
YYYi Yang
Yi Yang
WWWei Wang
Wei Wang
FLFuquan Liu
Fuquan Liu

15 December 2021
Verified
Conference Paper
Closed access
ATAnni Tang
Anni Tang
HXHan Xue
Han Xue
JLJun Ling
Jun Ling
RXRong Xie
Rong Xie
LSLi Sang
Li Sang
In face synthesis tasks, commonly used 2D face representations (e.g. 2D landmarks, segmentation maps, etc.) are usually sparse and discontinuous. To combat these shortcomings, we utilize a dense and continuous representation, named Projected Normalized Coordinate Code (PNCC), as the guidance and develop a PNCC-Spatio-Normalization (PSN) method to achieve face synthesis regarding arbitrary head poses and expressions. Based on PSN, we provide an effective framework for face reenactment and face swapping task. To ensure a harmonious and seamless face swapping, a simple yet effective Appearance-Blending Module (ABM) is proposed to fit the synthesized face to the target face. Our method is subject-agnostic and can be applied to any pair of faces without extra fine-tuning. Both qualitative and quantitative experiments are conducted to demonstrate the superiority of the proposed method in comparisons to existing state-of-the-art systems.

27 February 2026 in ACM Transactions on Multimedia Computing, Communications, and Applications
Verified
Journal Article
Open access
JLJun Ling
Jun Ling
YWYiwen Wang
Yiwen Wang
HXHan Xue
Han Xue
RXRong Xie
Rong Xie
LSLi Song
Li Song
Although audio-driven talking face generation has witnessed significant advances in recent years, two problems remain to be solved. First, the existing models cannot control the long-term head actions as humans expect, because the audio can only provide short-term cues, such as the rhythms and sentiments, for head movements. Second, generating long-term head poses and ensuring accurate lip motions remain challenging due to the difficulty in harnessing the optimization process for large-scale head movements and small-scale mouth motions. In this study, we propose a novel method to address these issues. First, to alleviate the limitations of audio conditions, we propose a Pose Latent Diffusion (PLD) model to generate head motions from two kinds of input modalities: the input audio and user-controlled text prompts. The audio provides short-term rhythm correspondence with the head movements, while the text prompts describe the long-term semantics of head motions. Second, we propose a refinement-based learning strategy to synthesize head movements and accurate lip motions using two cascaded networks, namely CoarseNet and RefineNet. The CoarseNet estimates coarse global motions to produce animated images with changed poses, and the RefineNet progressively estimates finer lip motions from low to high resolutions, yielding improved lip-synchronization performance. Experiments demonstrate that our method can achieve better pose diversity and realness compared to audio-based pose generation baselines, and our video generator model outperforms state-of-the-art methods in synthesizing natural head motions. Projects and demos are available at https://junleen.github.io/projects/posetalk .