Jiapeng Zhu

Jiapeng Zhu

Auto-generated profile, based on publicly available information

Jiapeng Zhu

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 November 2020
Verified
Book Chapter
Closed access
PDF file
JZJiapeng Zhu
Jiapeng Zhu
YSYujun Shen
Yujun Shen
DZDeli Zhao
Deli Zhao
BZBolei Zhou
Bolei Zhou

1 February 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
JZJiapeng Zhu
Jiapeng Zhu
YSYujun Shen
Yujun Shen
YXYinghao Xu
Yinghao Xu
DZDeli Zhao
Deli Zhao
QCQifeng Chen
Qifeng Chen
BZBolei Zhou
Bolei Zhou
Generative Adversarial Networks (GANs) have significantly advanced image synthesis through mapping randomly sampled latent codes to high-fidelity synthesized images. However, applying well-trained GANs to real image editing remains challenging. A common solution is to find an approximate latent code that can adequately recover the input image to edit, which is also known as GAN inversion. To invert a GAN model, prior works typically focus on reconstructing the target image at the pixel level, yet few studies are conducted on whether the inverted result can well support manipulation at the semantic level. This work fills in this gap by proposing in-domain GAN inversion , which consists of a domain-guided encoder and a domain-regularized optimizer , to regularize the inverted code in the native latent space of the pre-trained GAN model. In this way, we manage to sufficiently reuse the knowledge learned by GANs for image reconstruction, facilitating a wide range of editing applications without any retraining. We further make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property. Such a trade-off sheds light on how a GAN model represents an image with various semantics encoded in the learned latent distribution.

1 December 2022 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YXYinghao Xu
Yinghao Xu
YSYujun Shen
Yujun Shen
JZJiapeng Zhu
Jiapeng Zhu
CYCeyuan Yang
Ceyuan Yang
BZBolei Zhou
Bolei Zhou
Recent years witness the tremendous success of generative adversarial networks (GANs) in synthesizing photo-realistic images. GAN generator learns to compose realistic images and reproduce the real data distribution. Through that, a hierarchical visual feature with multi-level semantics spontaneously emerges. In this work we investigate that such a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks, including both generative ones and more importantly discriminative ones. We first train an encoder by considering the pre-trained StyleGAN generator as a learned loss function. The visual features produced by our encoder, termed as Generative Hierarchical Features (GH-Feat) , highly align with the layer-wise GAN representations, and hence describe the input image adequately from the reconstruction perspective. Extensive experiments support the versatile transferability of GH-Feat across a range of applications, such as image editing, image processing, image harmonization, face verification, landmark detection, layout prediction, image retrieval, etc. We further show that, through a proper spatial expansion, our developed GH-Feat can also facilitate fine-grained semantic segmentation using only a few annotations. Both qualitative and quantitative results demonstrate the appealing performance of GH-Feat. Code and models are available at https://genforce.github.io/ghfeat/ .

29 October 2024
Verified
Book Chapter
Closed access
PDF file
YZYifei Zhang
Yifei Zhang
MXMengfei Xia
Mengfei Xia
YSYujun Shen
Yujun Shen
JZJiapeng Zhu
Jiapeng Zhu
CYCeyuan Yang
Ceyuan Yang
Fan ChengFan Cheng
Fan Cheng