Abhimitra Meka

Abhimitra Meka

Auto-generated profile, based on publicly available information

Abhimitra Meka

Google Inc., United States of America

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 July 2023
Verified
Conference Paper
Open access
XPXingang Pan
Xingang Pan
ATAyush Tewari
Ayush Tewari
TLThomas Leimkühler
Thomas Leimkühler
LLLingjie Liu
Lingjie Liu
AMAbhimitra Meka
Abhimitra Meka
CTChristian Theobalt
Christian Theobalt
Synthesizing visual content that meets users’ needs often requires flexible and precise controllability of the pose, shape, expression, and layout of the generated objects. Existing approaches gain controllability of generative adversarial networks (GANs) via manually annotated training data or a prior 3D model, which often lack flexibility, precision, and generality. In this work, we study a powerful yet much less explored way of controlling GANs, that is, to "drag" any points of the image to precisely reach target points in a user-interactive manner, as shown in Fig.1. To achieve this, we propose DragGAN, which consists of two main components: 1) a feature-based motion supervision that drives the handle point to move towards the target position, and 2) a new point tracking approach that leverages the discriminative generator features to keep localizing the position of the handle points. Through DragGAN, anyone can deform an image with precise control over where pixels go, thus manipulating the pose, shape, expression, and layout of diverse categories such as animals, cars, humans, landscapes, etc. As these manipulations are performed on the learned generative image manifold of a GAN, they tend to produce realistic outputs even for challenging scenarios such as hallucinating occluded content and deforming shapes that consistently follow the object’s rigidity. Both qualitative and quantitative comparisons demonstrate the advantage of DragGAN over prior approaches in the tasks of image manipulation and point tracking. We also showcase the manipulation of real images through GAN inversion.

10 December 2023
Verified
Conference Paper
Open access
KSKripasindhu Sarkar
Kripasindhu Sarkar
MBMarcel C. Bühler
Marcel C. Bühler
GLGengyan Li
Gengyan Li
DWDaoye Wang
Daoye Wang
DVDelio Vicini
Delio Vicini
AMAbhimitra Meka
Abhimitra Meka
High-fidelity, photorealistic 3D capture of a human face is a long-standing problem in computer graphics – the complex material of skin, intricate geometry of hair, and fine scale textural details make it challenging. Traditional techniques rely on very large and expensive capture rigs to reconstruct explicit mesh geometry and appearance maps, and are limited by the accuracy of hand-crafted reflectance models. More recent volumetric methods (e.g., NeRFs) have enabled view-synthesis and sometimes relighting by learning an implicit representation of the density and reflectance basis, but suffer from artifacts and blurriness due to the inherent ambiguities in volumetric modeling. These problems are further exacerbated when capturing with few cameras and light sources. We present a novel technique for high-quality capture of a human face for 3D view synthesis and relighting using a sparse, compact capture rig consisting of 15 cameras and 15 lights. Our method combines a neural volumetric representation with traditional mesh reconstruction from multiview stereo. The proxy geometry allows us to anchor the 3D density field to prevent artifacts and guide the disentanglement of intrinsic radiance components of the face appearance such as diffuse and specular reflectance, and incident radiance (shadowing) fields. Our hybrid representation significantly improves the state-of-the-art quality for arbitrarily dense renders of a face from desired camera viewpoint as well as environmental, directional, and near-field lighting.

3 December 2024
Verified
Conference Paper
Open access
MBMarcel C. Buehler
Marcel C. Buehler
GLGengyan Li
Gengyan Li
EWErroll Wood
Erroll Wood
LHLeonhard Helminger
Leonhard Helminger
XCXu Chen
Xu Chen
KSKripasindhu Sarkar
Kripasindhu Sarkar

13 July 2024
Verified
Conference Paper
Open access
PRPramod Rao
Pramod Rao
GFGereon Fox
Gereon Fox
AMAbhimitra Meka
Abhimitra Meka
MRMallikarjun B R
Mallikarjun B R
FZFangneng Zhan
Fangneng Zhan
CTChristian Theobalt
Christian Theobalt
Achieving photorealistic 3D view synthesis and relighting of human portraits is pivotal for advancing AR/VR applications. Existing methodologies in portrait relighting demonstrate substantial limitations in terms of generalization and 3D consistency, coupled with inaccuracies in physically realistic lighting and identity preservation. Furthermore, personalization from a single view is difficult to achieve and often requires multiview images during the testing phase or involves slow optimization processes. This paper introduces Lite2Relight , a novel technique that can predict 3D consistent head poses of portraits while performing physically plausible light editing at interactive speed. Our method uniquely extends the generative capabilities and efficient volumetric representation of EG3D, leveraging a lightstage dataset to implicitly disentangle face reflectance and perform relighting under target HDRI environment maps. By utilizing a pre-trained geometry-aware encoder and a feature alignment module, we map input images into a relightable 3D space, enhancing them with a strong face geometry and reflectance prior. Through extensive quantitative and qualitative evaluations, we show that our method outperforms the state-of-the-art methods in terms of efficacy, photorealism, and practical application. This includes producing 3D-consistent results of the full head, including hair, eyes, and expressions. Lite2Relight paves the way for large-scale adoption of photorealistic portrait editing in various domains, offering a robust, interactive solution to a previously constrained problem.

24 April 2024 in Computer Graphics Forum
Verified
Journal Article
Open access
PDF file
GLG. Li
G. Li
KSK. Sarkar
K. Sarkar
AMA. Meka
A. Meka
MBM. Buehler
M. Buehler
FMF. Mueller
F. Mueller
TBT. Beeler
T. Beeler
Eye gaze and expressions are crucial non‐verbal signals in face‐to‐face communication. Visual effects and telepresence demand significant improvements in personalized tracking, animation, and synthesis of the eye region to achieve true immersion. Morphable face models, in combination with coordinate‐based neural volumetric representations, show promise in solving the difficult problem of reconstructing intricate geometry (eyelashes) and synthesizing photorealistic appearance variations (wrinkles and specularities) of eye performances. We propose a novel hybrid representation ‐ ShellNeRF ‐ that builds a discretized volume around a 3DMM face mesh using concentric surfaces to model the deformable ‘periocular’ region. We define a canonical space using the UV layout of the shells that constrains the space of dense correspondence search. Combined with an explicit eyeball mesh for modeling corneal light‐transport, our model allows for animatable photorealistic 3D synthesis of the whole eye region. Using multi‐view video input, we demonstrate significant improvements over state‐of‐the‐art in expression re‐enactment and transfer for high‐resolution close‐up views of the eye region.

14 December 2025
Verified
Conference Paper
Open access
PRPramod Rao
Pramod Rao
AMAbhimitra Meka
Abhimitra Meka
XZXilong Zhou
Xilong Zhou
GFGereon Fox
Gereon Fox
MRMallikarjun B R
Mallikarjun B R
CTChristian Theobalt
Christian Theobalt