Keqiang Sun

Keqiang Sun

Auto-generated profile, based on publicly available information

Keqiang Sun

CUHK MMLab

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 December 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
KSKeqiang Sun
Keqiang Sun
SWShangzhe Wu
Shangzhe Wu
NZNing Zhang
Ning Zhang
ZHZhaoyang Huang
Zhaoyang Huang
QWQuan Wang
Quan Wang
HLHongsheng Li
Hongsheng Li
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e.g., controlling the shapes, expressions, textures, and poses of the generated face images. However, previous methods focus on controllable 2D image generative models, which are prone to producing inconsistent face images under large expression and pose changes. In this paper, we propose a new NeRF-based conditional 3D face synthesis framework, which enables 3D controllability over the generated face images by imposing explicit 3D conditions from 3D face priors. At its core is a conditional Generative Occupancy Field (cGOF++) that effectively enforces the shape of the generated face to conform to a given 3D Morphable Model (3DMM) mesh, built on top of EG3D (Chan et al.~2022), a recent tri-plane-based generative model. To achieve accurate control over fine-grained 3D face shapes of the synthesized images, we additionally incorporate a 3D landmark loss as well as a volume warping loss into our synthesis framework. Experiments validate the effectiveness of the proposed method, which is able to generate high-fidelity face images and shows more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods.

19 November 2024
Verified
Conference Paper
Open access
FWFu-Yun Wang
Fu-Yun Wang
ZHZhaoyang Huang
Zhaoyang Huang
WBWeikang Bian
Weikang Bian
XSXiaoyu Shi
Xiaoyu Shi
KSKeqiang Sun
Keqiang Sun
HLHongsheng Li
Hongsheng Li

30 September 2024
Verified
Book Chapter
Closed access
PDF file
KSKeqiang Sun
Keqiang Sun
DLDor Litvak
Dor Litvak
YZYunzhi Zhang
Yunzhi Zhang
HLHongsheng Li
Hongsheng Li
JWJiajun Wu
Jiajun Wu
SWShangzhe Wu
Shangzhe Wu

29 October 2023 in arXiv
Verified
Preprint
Open access
PDF file
KSKeqiang Sun
Keqiang Sun
SWShangzhe Wu
Shangzhe Wu
NZNing Zhang
Ning Zhang
ZHZhaoyang Huang
Zhaoyang Huang
QWQuan Wang
Quan Wang
HLHongsheng Li
Hongsheng Li
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e.g., controlling the shapes, expressions, textures, and poses of the generated face images. However, previous methods focus on controllable 2D image generative models, which are prone to producing inconsistent face images under large expression and pose changes. In this paper, we propose a new NeRF-based conditional 3D face synthesis framework, which enables 3D controllability over the generated face images by imposing explicit 3D conditions from 3D face priors. At its core is a conditional Generative Occupancy Field (cGOF++) that effectively enforces the shape of the generated face to conform to a given 3D Morphable Model (3DMM) mesh, built on top of EG3D [1], a recent tri-plane-based generative model. To achieve accurate control over fine-grained 3D face shapes of the synthesized images, we additionally incorporate a 3D landmark loss as well as a volume warping loss into our synthesis framework. Experiments validate the effectiveness of the proposed method, which is able to generate high-fidelity face images and shows more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods.

16 May 2025 in arXiv
Verified
Preprint
Open access
PDF file
FWFu-Yun Wang
Fu-Yun Wang
YSYunhao Shui
Yunhao Shui
JPJingtan Piao
Jingtan Piao
KSKeqiang Sun
Keqiang Sun
HLHongsheng Li
Hongsheng Li
Diffusion models have made substantial advances in image generation, yet models trained on large, unfiltered datasets often yield outputs misaligned with human preferences. Numerous methods have been proposed to fine-tune pre-trained diffusion models, achieving notable improvements in aligning generated outputs with human preferences. However, we argue that existing preference alignment methods neglect the critical role of handling unconditional/negative-conditional outputs, leading to a diminished capacity to avoid generating undesirable outcomes. This oversight limits the efficacy of classifier-free guidance~(CFG), which relies on the contrast between conditional generation and unconditional/negative-conditional generation to optimize output quality. In response, we propose a straightforward but versatile effective approach that involves training a model specifically attuned to negative preferences. This method does not require new training strategies or datasets but rather involves minor modifications to existing techniques. Our approach integrates seamlessly with models such as SD1.5, SDXL, video diffusion models and models that have undergone preference optimization, consistently enhancing their alignment with human preferences.

12 December 2022 in arXiv
Verified
Preprint
Open access
PDF file
KSKeqiang Sun
Keqiang Sun
SWShangzhe Wu
Shangzhe Wu
ZHZhaoyang Huang
Zhaoyang Huang
NZNing Zhang
Ning Zhang
QWQuan Wang
Quan Wang
HLHongsheng Li
Hongsheng Li
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e.g., controlling the shapes, expressions, textures, and poses of the generated face images. However, these methods focus on 2D image generative models, which are prone to producing inconsistent face images under large expression and pose changes. In this paper, we propose a new NeRF-based conditional 3D face synthesis framework, which enables 3D controllability over the generated face images by imposing explicit 3D conditions from 3D face priors. At its core is a conditional Generative Occupancy Field (cGOF) that effectively enforces the shape of the generated face to commit to a given 3D Morphable Model (3DMM) mesh. To achieve accurate control over fine-grained 3D face shapes of the synthesized image, we additionally incorporate a 3D landmark loss as well as a volume warping loss into our synthesis algorithm. Experiments validate the effectiveness of the proposed method, which is able to generate high-fidelity face images and shows more precise 3D controllability than state-of-the-art 2D-based controllable face synthesis methods. Find code and demo at https://keqiangsun.github.io/projects/cgof.