Anqi Pang

Anqi Pang

Auto-generated profile, based on publicly available information

Anqi Pang

Shanghai Engineering Research Center of Intelligent Vision and Imaging School of Information Science and Technology ShanghaiTech University Shanghai China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 July 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
LZLongwen Zhang
Longwen Zhang
ZWZiyu Wang
Ziyu Wang
QZQixuan Zhang
Qixuan Zhang
QQQiwei Qiu
Qiwei Qiu
APAnqi Pang
Anqi Pang
JYJingyi Yu
Jingyi Yu
In the realm of digital creativity, our potential to craft intricate 3D worlds from imagination is often hampered by the limitations of existing digital tools, which demand extensive expertise and efforts. To narrow this disparity, we introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic text or image inputs as well as 3D-aware controls from diverse primitives (multi-view images, voxels, bounding boxes, point clouds, implicit representations, etc). At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT), to extract rich 3D priors directly from a diverse range of 3D geometries. Specifically, it adopts neural fields to represent continuous and complete surfaces and uses a geometry generative module with pure transformer blocks in latent space. We present a progressive training scheme to train CLAY on an ultra large 3D model dataset obtained through a carefully designed processing pipeline, resulting in a 3D native geometry generator with 1.5 billion parameters. For appearance generation, CLAY sets out to produce physically-based rendering (PBR) textures by employing a multi-view material diffusion model that can generate 2K resolution textures with diffuse, roughness, and metallic modalities. We demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details. Even first time users can easily use CLAY to bring their vivid 3D imaginations to life, unleashing unlimited creativity.

4 August 2021 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
XCXin Chen
Xin Chen
APAnqi Pang
Anqi Pang
WYWei Yang
Wei Yang
YMYuexin Ma
Yuexin Ma
LXLan Xu
Lan Xu
JYJingyi Yu
Jingyi Yu

16 July 2021 in arXiv
Verified
Preprint
Open access
PDF file
XCXin Chen
Xin Chen
APAnqi Pang
Anqi Pang
WYWei Yang
Wei Yang
YMYuexin Ma
Yuexin Ma
LXLan Xu
Lan Xu
JYJingyi Yu
Jingyi Yu
Markerless motion capture and understanding of professional non-daily human movements is an important yet unsolved task, which suffers from complex motion patterns and severe self-occlusion, especially for the monocular setting. In this paper, we propose SportsCap -- the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input. Our approach utilizes the semantic and temporally structured sub-motion prior in the embedding space for motion capture and understanding in a data-driven multi-task manner. To enable robust capture under complex motion patterns, we propose an effective motion embedding module to recover both the implicit motion embedding and explicit 3D motion details via a corresponding mapping function as well as a sub-motion classifier. Based on such hybrid motion information, we introduce a multi-stream spatial-temporal Graph Convolutional Network(ST-GCN) to predict the fine-grained semantic action attributes, and adopt a semantic attribute mapping block to assemble various correlated action attributes into a high-level action label for the overall detailed understanding of the whole sequence, so as to enable various applications like action assessment or motion scoring. Comprehensive experiments on both public and our proposed datasets show that with a challenging monocular sports video input, our novel approach not only significantly improves the accuracy of 3D human motion capture, but also recovers accurate fine-grained semantic action attributes.

28 September 2021 in arXiv
Verified
Preprint
Open access
PDF file
APAnqi Pang
Anqi Pang
XCXin Chen
Xin Chen
HLHaimin Luo
Haimin Luo
MWMinye Wu
Minye Wu
JYJingyi Yu
Jingyi Yu
LXLan Xu
Lan Xu
Recent neural rendering approaches for human activities achieve remarkable view synthesis results, but still rely on dense input views or dense training with all the capture frames, leading to deployment difficulty and inefficient training overload. However, existing advances will be ill-posed if the input is both spatially and temporally sparse. To fill this gap, in this paper we propose a few-shot neural human rendering approach (FNHR) from only sparse RGBD inputs, which exploits the temporal and spatial redundancy to generate photo-realistic free-view output of human activities. Our FNHR is trained only on the key-frames which expand the motion manifold in the input sequences. We introduce a two-branch neural blending to combine the neural point render and classical graphics texturing pipeline, which integrates reliable observations over sparse key-frames. Furthermore, we adopt a patch-based adversarial training process to make use of the local redundancy and avoids over-fitting to the key-frames, which generates fine-detailed rendering results. Extensive experiments demonstrate the effectiveness of our approach to generate high-quality free view-point results for challenging human performances under the sparse setting.