Chao Liu

Chao Liu

Auto-generated profile, based on publicly available information

Chao Liu

NVIDIA, Santa Clara, United States of America

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 July 2023 in ACM Transactions on Graphics
Verified
Journal Article
Open access
ATAlex Trevithick
Alex Trevithick
MCMatthew Chan
Matthew Chan
MSMichael Stengel
Michael Stengel
ECEric Chan
Eric Chan
CLChao Liu
Chao Liu
KNKoki Nagano
Koki Nagano
We present a one-shot method to infer and render a photorealistic 3D representation from a single unposed image (e.g., face portrait) in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher quality results than strong GAN-inversion baselines that require test-time optimization. To train our triplane encoder pipeline, we use only synthetic data, showing how to distill the knowledge from a pretrained 3D GAN into a feedforward encoder. Technical contributions include a Vision Transformer-based triplane encoder, a camera data augmentation strategy, and a well-designed loss function for synthetic data training. We benchmark against the state-of-the-art methods, demonstrating significant improvements in robustness and image quality in challenging real-world settings. We showcase our results on portraits of faces (FFHQ) and cats (AFHQ), but our algorithm can also be applied in the future to other categories with a 3D-aware image generator.

1 October 2023
Verified
Book Chapter
Closed access
PDF file
BOBatu Ozturkler
Batu Ozturkler
CLChao Liu
Chao Liu
BEBenjamin Eckart
Benjamin Eckart
MMMorteza Mardani
Morteza Mardani
JSJiaming Song
Jiaming Song
JKJan Kautz
Jan Kautz

26 July 2023
Verified
Conference Paper
Open access
MSMichael Stengel
Michael Stengel
KNKoki Nagano
Koki Nagano
CLChao Liu
Chao Liu
MCMatthew Chan
Matthew Chan
ATAlex Trevithick
Alex Trevithick
DLDavid Luebke
David Luebke
We present an AI-mediated 3D video conferencing system that can reconstruct and autostereoscopically display a life-sized talking head using consumer-grade compute resources and minimal capture equipment. Our 3D capture uses a novel 3D lifting method that encodes a given 2D input into an efficient triplanar neural representation of the user, which can be rendered from novel viewpoints in real-time. Our AI-based techniques drastically reduce the cost for 3D capture, while providing a high-fidelity 3D representation on the receiver’s end at the cost of traditional 2D video streaming. Additional advantages of our AI-based approach include the ability to accommodate both photorealistic and stylized avatars, and the ability to enable mutual eye contact in multi-directional video conferencing. We demonstrate our system using a tracked stereo display for a personal viewing experience as well as a lightfield display for a room-scale multi-viewer experience.

3 December 2024
Verified
Conference Paper
Open access
CLChao Liu
Chao Liu
WNWeili Nie
Weili Nie
SLSifei Liu
Sifei Liu
ABAbhishek Badki
Abhishek Badki
HSHang Su
Hang Su
AVArash Vahdat
Arash Vahdat

12 May 2025
Verified
Book Chapter
Closed access
PDF file
SLSifei Liu
Sifei Liu
WNWeili Nie
Weili Nie
ACAn-Chieh Cheng
An-Chieh Cheng
MMMorteza Mardani
Morteza Mardani
CLChao Liu
Chao Liu
AVArash Vahdat
Arash Vahdat