Vasu Agrawal

Vasu Agrawal

Auto-generated profile, based on publicly available information

Vasu Agrawal

Codec Avatars Lab, Meta, Pittsburgh, USA

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 December 2023
Verified
Conference Paper
Open access
LXLinning Xu
Linning Xu
VAVasu Agrawal
Vasu Agrawal
WLWilliam Laney
William Laney
TGTony Garcia
Tony Garcia
ABAayush Bansal
Aayush Bansal
CRChristian Richardt
Christian Richardt
We present an end-to-end system for the high-fidelity capture, model reconstruction, and real-time rendering of walkable spaces in virtual reality using neural radiance fields. To this end, we designed and built a custom multi-camera rig to densely capture walkable spaces in high fidelity and with multi-view high dynamic range images in unprecedented quality and density. We extend instant neural graphics primitives with a novel perceptual color space for learning accurate HDR appearance, and an efficient mip-mapping mechanism for level-of-detail rendering with anti-aliasing, while carefully optimizing the trade-off between quality and speed. Our multi-GPU renderer enables high-fidelity volume rendering of our neural radiance field model at the full VR resolution of dual 2K × 2K at 36 Hz on our custom demo machine. We demonstrate the quality of our results on our challenging high-fidelity datasets, and compare our method and datasets to existing baselines. We release our dataset on our project website: https://vr-nerf.github.io.

1 May 2022 in Field Robotics
Verified
Journal Article
Open access
SSSebastian Scherer
Sebastian Scherer
VAVasu Agrawal
Vasu Agrawal
GBGraeme Best
Graeme Best
CCChao Cao
Chao Cao
KCKatarina Cujic
Katarina Cujic
MTMatthew Travers
Matthew Travers

27 July 2025
Verified
Conference Paper
Open access
SWShaofei Wang
Shaofei Wang
TSTomas Simon
Tomas Simon
ISIgor Santesteban
Igor Santesteban
TBTimur Bagautdinov
Timur Bagautdinov
JLJunxuan Li
Junxuan Li
SSShunsuke Saito
Shunsuke Saito

31 March 2022 in arXiv
Verified
Preprint
Open access
PDF file
KYKarren Yang
Karren Yang
DMDejan Markovic
Dejan Markovic
SKSteven Krenn
Steven Krenn
VAVasu Agrawal
Vasu Agrawal
ARAlexander Richard
Alexander Richard
Since facial actions such as lip movements contain significant information about speech content, it is not surprising that audio-visual speech enhancement methods are more accurate than their audio-only counterparts. Yet, state-of-the-art approaches still struggle to generate clean, realistic speech without noise artifacts and unnatural distortions in challenging acoustic environments. In this paper, we propose a novel audio-visual speech enhancement framework for high-fidelity telecommunications in AR/VR. Our approach leverages audio-visual speech cues to generate the codes of a neural speech codec, enabling efficient synthesis of clean, realistic speech from noisy signals. Given the importance of speaker-specific cues in speech, we focus on developing personalized models that work well for individual speakers. We demonstrate the efficacy of our approach on a new audio-visual speech dataset collected in an unconstrained, large vocabulary setting, as well as existing audio-visual datasets, outperforming speech enhancement baselines on both quantitative metrics and human evaluation studies. Please see the supplemental video for qualitative results at https://github.com/facebookresearch/facestar/releases/download/paper_materials/video.mp4.