Kunlin Liu

Mr. Kunlin Liu


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 June 2021 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
KLKunlin Liu
Kunlin Liu
DCDongdong Chen
Dongdong Chen
JLJing Liao
Jing Liao
WZWeiming Zhang
Weiming Zhang
HZHang Zhou
Hang Zhou
NYNenghai Yu
Nenghai Yu
Invertible grayscale is a special kind of grayscale from which the original color can be recovered. Given an input color image, this seminal work tries to hide the color information into its grayscale counterpart while making it hard to recognize any anomalies. This powerful functionality is enabled by training a hiding sub-network and restoring sub-network in an end-to-end way. Despite its expressive results, two key limitations exist: 1) The restored color image often suffers from some noticeable visual artifacts in the smooth regions. 2) It is very sensitive to JPEG compression, i.e., the original color information cannot be well recovered once the intermediate grayscale image is compressed by JPEG. To overcome these two limitations, this article introduces adversarial training and JPEG simulator respectively. Specifically, two auxiliary adversarial networks are incorporated to make the intermediate grayscale images and final restored color images indistinguishable from normal grayscale and color images. And the JPEG simulator is utilized to simulate real JPEG compression during the online training so that the hiding and restoring sub-networks can automatically learn to be JPEG robust. Extensive experiments demonstrate that the proposed method is superior to the original invertible grayscale work both qualitatively and quantitatively while ensuring the JPEG robustness. We further show that the proposed framework can be applied under different types of grayscale constraints and achieve excellent results.

14 October 2022 in Signal Processing
Verified
Journal Article
Closed access
HLHonggu Liu
Honggu Liu
WZWenbo Zhou
Wenbo Zhou
DCDongdong Chen
Dongdong Chen
HFHan Fang
Han Fang
HBHuanyu Bian
Huanyu Bian
NYNenghai Yu
Nenghai Yu

29 June 2021 in arXiv
Verified
Preprint
Open access
PDF file
IPIvan Perov
Ivan Perov
DGDaiheng Gao
Daiheng Gao
NCNikolay Chervoniy
Nikolay Chervoniy
KLKunlin Liu
Kunlin Liu
SMSugasa Marangonda
Sugasa Marangonda
WZWeiming Zhang
Weiming Zhang
Deepfake defense not only requires the research of detection but also requires the efforts of generation methods. However, current deepfake methods suffer the effects of obscure workflow and poor performance. To solve this problem, we present DeepFaceLab, the current dominant deepfake framework for face-swapping. It provides the necessary tools as well as an easy-to-use way to conduct high-quality face-swapping. It also offers a flexible and loose coupling structure for people who need to strengthen their pipeline with other features without writing complicated boilerplate code. We detail the principles that drive the implementation of DeepFaceLab and introduce its pipeline, through which every aspect of the pipeline can be modified painlessly by users to achieve their customization purpose. It is noteworthy that DeepFaceLab could achieve cinema-quality results with high fidelity. We demonstrate the advantage of our system by comparing our approach with other face-swapping methods.For more information, please visit:https://github.com/iperov/DeepFaceLab/.

22 November 2021 in Future Internet
Verified
Journal Article
Open access
PDF file
KLKunlin Liu
Kunlin Liu
PWPing Wang
Ping Wang
WZWenbo Zhou
Wenbo Zhou
ZZZhenyu Zhang
Zhenyu Zhang
YGYanhao Ge
Yanhao Ge
NYNenghai Yu
Nenghai Yu
Deepfake aims to swap a face of an image with someone else’s likeness in a reasonable manner. Existing methods usually perform deepfake frame by frame, thus ignoring video consistency and producing incoherent results. To address such a problem, we propose a novel framework Neural Identity Carrier (NICe), which learns identity transformation from an arbitrary face-swapping proxy via a U-Net. By modeling the incoherence between frames as noise, NICe naturally suppresses its disturbance and preserves primary identity information. Concretely, NICe inputs the original frame and learns transformation supervised by swapped pseudo labels. As the temporal incoherence has an uncertain or stochastic pattern, NICe can filter out such outliers and well maintain the target content by uncertainty prediction. With the predicted temporally stable appearance, NICe enhances its details by constraining 3D geometry consistency, making NICe learn fine-grained facial structure across the poses. In this way, NICe guarantees the temporal stableness of deepfake approaches and predicts detailed results against over-smoothness. Extensive experiments on benchmarks demonstrate that NICe significantly improves the quality of existing deepfake methods on video-level. Besides, data generated by our methods can benefit video-level deepfake detection methods.

25 September 2023
Verified
Conference Paper
Closed access
BPBo Peng
Bo Peng
XSXianyun Sun
Xianyun Sun
Caiyong WangCaiyong Wang
Caiyong Wang
WWWei Wang
Wei Wang
JDJing Dong
Jing Dong
WDWenxiu Diao
Wenxiu Diao
This paper presents the summary report on the DeepFake Game Competition on Visual Realism Assessment (DFGC-VRA). Deep-learning based face-swap videos, also known as deepfakes, are becoming more and more realistic and deceiving. The malicious usage of these face-swap videos has caused wide concerns. There is a ongoing deepfake game between its creators and detectors, with the human in the loop. The research community has been focusing on the automatic detection of these fake videos, but the assessment of their visual realism, as perceived by human eyes, is still an unexplored dimension. Visual realism assessment, or VRA, is essential for assessing the potential impact that may be brought by a specific face-swap video, and it is also useful as a quality metric to compare different face-swap methods. This is the third edition of DFGC competitions, which focuses on the new visual realism assessment topic, different from previous ones that compete creators versus detectors. With this competition, we conduct a comprehensive study of the SOTA performance on the new task. We also release our MindSpore codes to further facilitate research in this field (https://github.com/bomb2peng/DFGC-VRA-benckmark).

11 January 2024 in Journal of Imaging
Verified
Journal Article
Open access
PDF file
XZXuehai Zhang
Xuehai Zhang
WZWenbo Zhou
Wenbo Zhou
KLKunlin Liu
Kunlin Liu
HTHao Tang
Hao Tang
ZZZhenyu Zhang
Zhenyu Zhang
NYNenghai Yu
Nenghai Yu
Face swapping is an intriguing and intricate task in the field of computer vision. Currently, most mainstream face swapping methods employ face recognition models to extract identity features and inject them into the generation process. Nonetheless, such methods often struggle to effectively transfer identity information, which leads to generated results failing to achieve a high identity similarity to the source face. Furthermore, if we can accurately disentangle identity information, we can achieve controllable face swapping, thereby providing more choices to users. In pursuit of this goal, we propose a new face swapping framework (ControlFace) based on the disentanglement of identity information. We disentangle the structure and texture of the source face, encoding and characterizing them in the form of feature embeddings separately. According to the semantic level of each feature representation, we inject them into the corresponding feature mapper and fuse them adequately in the latent space of StyleGAN. Owing to such disentanglement of structure and texture, we are able to controllably transfer parts of the identity features. Extensive experiments and comparisons with state-of-the-art face swapping methods demonstrate the superiority of our face swapping framework in terms of transferring identity information, producing high-quality face images, and controllable face swapping.