Fujun Luan

Fujun Luan

Auto-generated profile, based on publicly available information

Fujun Luan

Adobe Research, San Jose, USA

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 July 2024
Verified
Conference Paper
Open access
ZZZheng Zeng
Zheng Zeng
VDValentin Deschaintre
Valentin Deschaintre
IGIliyan Georgiev
Iliyan Georgiev
YHYannick Hold-Geoffroy
Yannick Hold-Geoffroy
YHYiwei Hu
Yiwei Hu
MHMiloš Hašan
Miloš Hašan
The three areas of realistic forward rendering, per-pixel inverse rendering, and generative image synthesis may seem like separate and unrelated sub-fields of graphics and vision. However, recent work has demonstrated improved estimation of per-pixel intrinsic channels (albedo, roughness, metallicity) based on a diffusion architecture; we call this the RGB → X problem. We further show that the reverse problem of synthesizing realistic images given intrinsic channels, X → RGB, can also be addressed in a diffusion framework. Focusing on the image domain of interior scenes, we introduce an improved diffusion model for RGB → X, which also estimates lighting, as well as the first diffusion X → RGB model capable of synthesizing realistic images from (full or partial) intrinsic channels. Our X → RGB model explores a middle ground between traditional rendering and generative models: We can specify only certain appearance properties that should be followed, and give freedom to the model to hallucinate a plausible version of the rest. This flexibility allows using a mix of heterogeneous training datasets that differ in the available channels. We use multiple existing datasets and extend them with our own synthetic and real data, resulting in a model capable of extracting scene properties better than previous work and of generating highly realistic images of interior scenes.

23 July 2023
Verified
Conference Paper
Open access
BXBing Xu
Bing Xu
LWLiwen Wu
Liwen Wu
MHMilos Hasan
Milos Hasan
FLFujun Luan
Fujun Luan
IGIliyan Georgiev
Iliyan Georgiev
RRRavi Ramamoorthi
Ravi Ramamoorthi
Neural material representations have recently been proposed to augment the material appearance toolbox used in realistic rendering. These models are successful at tasks ranging from measured BTF compression, through efficient rendering of synthetic displaced materials with occlusions, to BSDF layering. However, importance sampling has been an after-thought in most neural material approaches, and has been handled by inefficient cosine-hemisphere sampling or mixing it with an additional simple analytic lobe. In this paper we fill that gap, by evaluating and comparing various pdf-learning approaches for sampling spatially varying neural materials, and proposing new variations of these approaches. We investigate three sampling approaches: analytic-lobe mixtures, normalizing flows, and histogram prediction. Within each type, we introduce improvements beyond previous work, and we extensively evaluate and compare these approaches in terms of sampling rate, wall-clock time, and final visual quality. Our versions of normalizing flows and histogram mixtures perform well and can be used in practical rendering systems, potentially facilitating the broader adoption of neural material models in production.

10 December 2023
Verified
Conference Paper
Open access
KYKai Yan
Kai Yan
FLFujun Luan
Fujun Luan
MHMiloš Hašan
Miloš Hašan
TGThibault Groueix
Thibault Groueix
VDValentin Deschaintre
Valentin Deschaintre
SZShuang Zhao
Shuang Zhao
A 3D digital scene contains many components: lights, materials and geometries, interacting to reach the desired appearance. Staging such a scene is time-consuming and requires both artistic and technical skills. In this work, we propose PSDR-Room, a system allowing to optimize lighting as well as the pose and materials of individual objects to match a target image of a room scene, with minimal user input. To this end, we leverage a recent path-space differentiable rendering approach that provides unbiased gradients of the rendering with respect to geometry, lighting, and procedural materials, allowing us to optimize all of these components using gradient descent to visually match the input photo appearance. We use recent single-image scene understanding methods to initialize the optimization and search for appropriate 3D models and materials. We evaluate our method on real photographs of indoor scenes and demonstrate the editability of the resulting scene components.

10 December 2023
Verified
Conference Paper
Open access
KGKunal Gupta
Kunal Gupta
MHMilos Hasan
Milos Hasan
ZXZexiang Xu
Zexiang Xu
FLFujun Luan
Fujun Luan
KSKalyan Sunkavalli
Kalyan Sunkavalli
SBSai Bi
Sai Bi
The volume rendering step used in Neural Radiance Fields (NeRFs) produces highly photorealistic results, but is inherently slow because it evaluates an MLP at a large number of sample points per ray. Previous work has addressed this by either proposing neural scene representations that are faster to evaluate or by pre-computing (and approximating) scene properties to reduce render times. In this work, we propose MCNeRF, a general Monte Carlo-based rendering algorithm that can speed up any NeRF representation. We show that the NeRF volume rendering integral can be efficiently computed via Monte Carlo integration using an importance sampling scheme based on ray density distributions. This allows us to use a small number of MLP evaluations to estimate pixel radiance. These noisy Monte Carlo estimates can be further denoised using an inexpensive image-space denoiser trained per-scene. We demonstrate that MCNeRF can be used to speed up NeRF representations like TensoRF by 7 × while closely matching their visual quality and without making the scene approximations that real-time NeRF rendering methods usually make.

1 November 2024
Verified
Book Chapter
Closed access
PDF file
SRSara Rojas
Sara Rojas
JPJulien Philip
Julien Philip
KZKai Zhang
Kai Zhang
SBSai Bi
Sai Bi
FLFujun Luan
Fujun Luan
KSKalyan Sunkavalli
Kalyan Sunkavalli

27 July 2025 in ACM Transactions on Graphics
Verified
Journal Article
Open access
XMXiaohe Ma
Xiaohe Ma
VDValentin Deschaintre
Valentin Deschaintre
MHMiloš Hašan
Miloš Hašan
FLFujun Luan
Fujun Luan
KZKun Zhou
Kun Zhou
YHYiwei Hu
Yiwei Hu
High-quality material generation is key for virtual environment authoring and inverse rendering. We propose MaterialPicker, a multi-modal material generator leveraging a Diffusion Transformer (DiT) architecture, improving and simplifying the creation of high-quality materials from text prompts and/or photographs. Our method can generate a material based on an image crop of a material sample, even if the captured surface is distorted, viewed at an angle or partially occluded, as is often the case in photographs of natural scenes. We further allow the user to specify a text prompt to provide additional guidance for the generation. We finetune a pre-trained DiT-based video generator into a material generator, where each material map is treated as a frame in a video sequence. We evaluate our approach both quantitatively and qualitatively and show that it enables more diverse material generation and better distortion correction than previous work.