Rongfei Jia

Rongfei Jia

Auto-generated profile, based on publicly available information

Rongfei Jia

Alibaba Group, Hangzhou, Zhejiang, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 December 2024
Verified
Conference Paper
Open access
YDYucheng Ding
Yucheng Ding
CNChaoyue Niu
Chaoyue Niu
YYYikai Yan
Yikai Yan
Zhenzhe ZhengZhenzhe Zheng
Zhenzhe Zheng
FWFan Wu
Fan Wu
RJRongfei Jia
Rongfei Jia

27 March 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
BCBowen Cai
Bowen Cai
YLYujie Li
Yujie Li
YLYuqin Liang
Yuqin Liang
RJRongfei Jia
Rongfei Jia
BZBinqiang Zhao
Binqiang Zhao
HFHuan Fu
Huan Fu
This paper studies how to flexibly integrate reconstructed 3D models into practical 3D modeling pipelines such as 3D scene creation and rendering. Due to the technical difficulty, one can only obtain rough 3D models (R3DMs) for most real objects using existing 3D reconstruction techniques. As a result, physically-based rendering (PBR) would render low-quality images or videos for scenes that are constructed by R3DMs. One promising solution would be representing real-world objects as Neural Fields such as NeRFs, which are able to generate photo-realistic renderings of an object under desired viewpoints. However, a drawback is that the synthesized views through Neural Fields Rendering (NFR) cannot reflect the simulated lighting details on R3DMs in PBR pipelines, especially when object interactions in the 3D scene creation cause local shadows. To solve this dilemma, we propose a lighting transfer network (LighTNet) to bridge NFR and PBR, such that they can benefit from each other. LighTNet reasons about a simplified image composition model, remedies the uneven surface issue caused by R3DMs, and is empowered by several perceptual-motivated constraints and a new Lab angle loss which enhances the contrast between lighting strength and colors. Comparisons demonstrate that LighTNet is superior in synthesizing impressive lighting, and is promising in pushing NFR further in practical 3D modeling workflows.

14 April 2022 in arXiv
Verified
Preprint
Open access
PDF file
YZYuanqing Zhang
Yuanqing Zhang
JSJiaming Sun
Jiaming Sun
XHXingyi He
Xingyi He
HFHuan Fu
Huan Fu
RJRongfei Jia
Rongfei Jia
XZXiaowei Zhou
Xiaowei Zhou
Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is rarely modeled in previous methods, as it requires expensive recursive path tracing which makes the inverse rendering computationally intractable. In this paper, we propose a novel approach to efficiently recovering spatially-varying indirect illumination. The key insight is that indirect illumination can be conveniently derived from the neural radiance field learned from input images instead of being estimated jointly with direct illumination and materials. By properly modeling the indirect illumination and visibility of direct illumination, interreflection- and shadow-free albedo can be recovered. The experiments on both synthetic and real data demonstrate the superior performance of our approach compared to previous work and its capability to synthesize realistic renderings under novel viewpoints and illumination. Our code and data are available at https://zju3dv.github.io/invrender/.

12 May 2022 in arXiv
Verified
Preprint
Open access
PDF file
JZJian Zhang
Jian Zhang
YZYuanqing Zhang
Yuanqing Zhang
HFHuan Fu
Huan Fu
XZXiaowei Zhou
Xiaowei Zhou
BCBowen Cai
Bowen Cai
XTXing Tang
Xing Tang
Neural Radiance Fields (NeRF) have emerged as a potent paradigm for representing scenes and synthesizing photo-realistic images. A main limitation of conventional NeRFs is that they often fail to produce high-quality renderings under novel viewpoints that are significantly different from the training viewpoints. In this paper, instead of exploiting few-shot image synthesis, we study the novel view extrapolation setting that (1) the training images can well describe an object, and (2) there is a notable discrepancy between the training and test viewpoints' distributions. We present RapNeRF (RAy Priors) as a solution. Our insight is that the inherent appearances of a 3D surface's arbitrary visible projections should be consistent. We thus propose a random ray casting policy that allows training unseen views using seen views. Furthermore, we show that a ray atlas pre-computed from the observed rays' viewing directions could further enhance the rendering quality for extrapolated views. A main limitation is that RapNeRF would remove the strong view-dependent effects because it leverages the multi-view consistency property.