Jiahui Lei

Jiahui Lei

Auto-generated profile, based on publicly available information

Jiahui Lei

University of Pennsylvania

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications

28 October 2024
Verified
Book Chapter
Closed access
PDF file
YSYunzhou Song
Yunzhou Song
JLJiahui Lei
Jiahui Lei
ZWZiyun Wang
Ziyun Wang
LLLingjie Liu
Lingjie Liu
KDKostas Daniilidis
Kostas Daniilidis

10 November 2025 in arXiv
Verified
Preprint
Open access
PDF file
LMLinzhan Mou
Linzhan Mou
JLJiahui Lei
Jiahui Lei
CWChen Wang
Chen Wang
LLLingjie Liu
Lingjie Liu
KDKostas Daniilidis
Kostas Daniilidis
We present DIMO, a generative approach capable of generating diverse 3D motions for arbitrary objects from a single image. The core idea of our work is to leverage the rich priors in well-trained video models to extract the common motion patterns and then embed them into a shared low-dimensional latent space. Specifically, we first generate multiple videos of the same object with diverse motions. We then embed each motion into a latent vector and train a shared motion decoder to learn the distribution of motions represented by a structured and compact motion representation, i.e., neural key point trajectories. The canonical 3D Gaussians are then driven by these key points and fused to model the geometry and appearance. During inference time with learned latent space, we can instantly sample diverse 3D motions in a single-forward pass and support several interesting applications including 3D motion interpolation and language-guided motion generation. Our project page is available at https://linzhanm.github.io/dimo.

7 June 2023 in arXiv
Verified
Preprint
Open access
PDF file
YXYinshuang Xu
Yinshuang Xu
JLJiahui Lei
Jiahui Lei
KDKostas Daniilidis
Kostas Daniilidis
3D reconstruction and novel view rendering can greatly benefit from geometric priors when the input views are not sufficient in terms of coverage and inter-view baselines. Deep learning of geometric priors from 2D images often requires each image to be represented in a $2D$ canonical frame and the prior to be learned in a given or learned $3D$ canonical frame. In this paper, given only the relative poses of the cameras, we show how to learn priors from multiple views equivariant to coordinate frame transformations by proposing an $SE(3)$-equivariant convolution and transformer in the space of rays in 3D. This enables the creation of a light field that remains equivariant to the choice of coordinate frame. The light field as defined in our work, refers both to the radiance field and the feature field defined on the ray space. We model the ray space, the domain of the light field, as a homogeneous space of $SE(3)$ and introduce the $SE(3)$-equivariant convolution in ray space. Depending on the output domain of the convolution, we present convolution-based $SE(3)$-equivariant maps from ray space to ray space and to $\mathbb{R}^3$. Our mathematical framework allows us to go beyond convolution to $SE(3)$-equivariant attention in the ray space. We demonstrate how to tailor and adapt the equivariant convolution and transformer in the tasks of equivariant neural rendering and $3D$ reconstruction from multiple views. We demonstrate $SE(3)$-equivariance by obtaining robust results in roto-translated datasets without performing transformation augmentation.