Yujun Shen

Yujun Shen

Auto-generated profile, based on publicly available information

Yujun Shen

Ant Group

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 October 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YSYujun Shen
Yujun Shen
CYCeyuan Yang
Ceyuan Yang
XTXiaoou Tang
Xiaoou Tang
BZBolei Zhou
Bolei Zhou
Although Generative Adversarial Networks (GANs) have made significant progress in face synthesis, there lacks enough understanding of what GANs have learned in the latent representation to map a randomly sampled code to a photo-realistic face image. In this work, we propose a framework, called InterFaceGAN, to interpret the disentangled face representation learned by the state-of-the-art GAN models and thoroughly analyze the properties of the facial semantics in the latent space. We first find that GANs actually learn various semantics in some linear subspaces of the latent space when being trained to synthesize high-quality faces. After identifying the subspaces of the corresponding latent semantics, we are able to realistically manipulate the facial attributes occurring in the synthesized images without retraining the model. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection, resulting in more precise control of the attribute manipulation. Besides manipulating gender, age, expression, and the presence of eyeglasses, we can even alter the face pose as well as fix the artifacts accidentally generated by GANs. Extensive experimental results suggest that learning to synthesize faces spontaneously brings a disentangled and controllable face representation.

19 November 2020
Verified
Book Chapter
Closed access
PDF file
JZJiapeng Zhu
Jiapeng Zhu
YSYujun Shen
Yujun Shen
DZDeli Zhao
Deli Zhao
BZBolei Zhou
Bolei Zhou

30 September 2024
Verified
Book Chapter
Closed access
PDF file
KZKecheng Zheng
Kecheng Zheng
Fan ChengFan Cheng
Fan Cheng
WWWei Wu
Wei Wu
FLFan Lu
Fan Lu
SMShuailei Ma
Shuailei Ma
YSYujun Shen
Yujun Shen

13 July 2024
Verified
Conference Paper
Open access
SZShangzhan Zhang
Shangzhan Zhang
SPSida Peng
Sida Peng
TXTao Xu
Tao Xu
YYYuanbo Yang
Yuanbo Yang
TCTianrun Chen
Tianrun Chen
XZXiaowei Zhou
Xiaowei Zhou
This paper aims to generate materials for 3D meshes from text descriptions. Unlike existing methods that synthesize texture maps, we propose to generate segment-wise procedural material graphs as the appearance representation, which supports high-quality rendering and provides substantial flexibility in editing. Instead of relying on extensive paired data, i.e., 3D meshes with material graphs and corresponding text descriptions, to train a material graph generative model, we propose to leverage the pre-trained 2D diffusion model as a bridge to connect the text and material graphs. Specifically, our approach decomposes a shape into a set of segments and designs a segment-controlled diffusion model to synthesize 2D images that are aligned with mesh parts. Based on generated images, we initialize parameters of material graphs and fine-tune them through the differentiable rendering module to produce materials in accordance with the textual description. Extensive experiments demonstrate the superior performance of our framework in photorealism, resolution, and editability over existing methods.

13 July 2024
Verified
Conference Paper
Open access
BZBoming Zhao
Boming Zhao
YLYuan Li
Yuan Li
ZSZiyu Sun
Ziyu Sun
LZLin Zeng
Lin Zeng
YSYujun Shen
Yujun Shen
ZCZhaopeng Cui
Zhaopeng Cui
Forecasting future scenarios in dynamic environments is essential for intelligent decision-making and navigation, a challenge yet to be fully realized in computer vision and robotics. Traditional approaches like video prediction and novel-view synthesis either lack the ability to forecast from arbitrary viewpoints or to predict temporal dynamics. In this paper, we introduce GaussianPrediction, a novel framework that empowers 3D Gaussian representations with dynamic scene modeling and future scenario synthesis in dynamic environments. GaussianPrediction can forecast future states from any viewpoint, using video observations of dynamic scenes. To this end, we first propose a 3D Gaussian canonical space with deformation modeling to capture the appearance and geometry of dynamic scenes, and integrate the lifecycle property into Gaussians for irreversible deformations. To make the prediction feasible and efficient, a concentric motion distillation approach is developed by distilling the scene motion with key points. Finally, a Graph Convolutional Network is employed to predict the motions of key points, enabling the rendering of photorealistic images of future scenarios. Our framework shows outstanding performance on both synthetic and real-world datasets, demonstrating its efficacy in predicting and rendering future environments. Code is available on the project webpage: https://zju3dv.github.io/gaussian-prediction.

27 July 2025 in ACM Transactions on Graphics
Verified
Journal Article
Open access
PDPinxuan Dai
Pinxuan Dai
PZPeiquan Zhang
Peiquan Zhang
ZDZheng Dong
Zheng Dong
KXKe Xu
Ke Xu
YPYifan Peng
Yifan Peng
WXWeiwei Xu
Weiwei Xu
Online free-view navigation in volumetric videos requires high-quality rendering and real-time streaming in order to provide immersive user experiences. However, existing methods ( e.g. , dynamic NeRF and 3DGS) may not handle dynamic scenes with complex motions, and their models may not be streamable due to storage and bandwidth constraints. In this paper, we propose a novel 4D Gaussian Video (4DGV) approach that enables the creation and streaming of photorealistic, volumetric videos for dynamic scenes over the Internet. The core of our 4DGV is a novel streamable group of Gaussians (GOG) representation based on motion layering. Each GOG consists of static and dynamic points obtained via lifting 2D segmentation into 3D in motion layering, where the deformation of each dynamic point is represented as the temporal offset of its attributes. We also adaptively convert static points back to dynamic points to handle the appearance change, (e.g. , moving shadows and reflections), of static objects through optimization. To support real-time streaming of 4DGVs, we show that by applying quantization on Gaussian attributes and H.265 encoding on deformation offsets, our GOG representation can be significantly compressed (to around 6% of the original model size) without sacrificing the accuracy (PSNR loss less than 0.01dB). Extensive experiments on standard benchmarks demonstrate that our method outperforms state-of-the-art volumetric video approaches, with superior rendering quality and minimum storage overheads.