Zhaopeng Cui

Zhaopeng Cui

Auto-generated profile, based on publicly available information

Zhaopeng Cui

Zhejiang University

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 October 2024
Verified
Book Chapter
Closed access
PDF file
JHJiarui Hu
Jiarui Hu
XCXianhao Chen
Xianhao Chen
BFBoyin Feng
Boyin Feng
GLGuanglin Li
Guanglin Li
LYLiangjing Yang
Liangjing Yang
ZCZhaopeng Cui
Zhaopeng Cui

10 March 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
HZHongjia Zhai
Hongjia Zhai
XZXiyu Zhang
Xiyu Zhang
BZBoming Zhao
Boming Zhao
HLHai Li
Hai Li
YHYijia He
Yijia He
GZGuofeng Zhang
Guofeng Zhang
Visual localization plays an important role in the applications of Augmented Reality (AR), which enable AR devices to obtain their 6-DoF pose in the pre-build map in order to render virtual content in real scenes. However, most existing approaches can not perform novel view rendering and require large storage capacities for maps. To overcome these limitations, we propose an efficient visual localization method capable of high-quality rendering with fewer parameters. Specifically, our approach leverages 3D Gaussian primitives as the scene representation. To ensure precise 2D–3D correspondences for pose estimation, we develop an unbiased 3D scene-specific descriptor decoder for Gaussian primitives, distilled from a constructed feature volume. Additionally, we introduce a salient 3D landmark selection algorithm that selects a suitable primitive subset based on the saliency score for localization. We further regularize key Gaussian primitives to prevent anisotropic effects, which also improves localization performance. Extensive experiments on two widely used datasets demonstrate that our method achieves superior or comparable rendering and localization performance to state-of-the-art implicit-based visual localization approaches. Code and data are available at project page: https://zju3dv.github.io/splatloc.

26 October 2023
Verified
Conference Paper
Open access
JZJunyi Zeng
Junyi Zeng
CBChong Bao
Chong Bao
RCRui Chen
Rui Chen
ZDZilong Dong
Zilong Dong
GZGuofeng Zhang
Guofeng Zhang
ZCZhaopeng Cui
Zhaopeng Cui
Recently, Neural Radiance Fields (NeRF) has exhibited significant success in novel view synthesis, surface reconstruction, etc. However, since no physical reflection is considered in its rendering pipeline, NeRF mistakes the reflection in the mirror as a separate virtual scene, leading to the inaccurate reconstruction of the mirror and multi-view inconsistent reflections in the mirror. In this paper, we present a novel neural rendering framework, named Mirror-NeRF, which is able to learn accurate geometry and reflection of the mirror and support various scene manipulation applications with mirrors, such as adding new objects or mirrors into the scene and synthesizing the reflections of these new objects in mirrors, controlling mirror roughness, etc. To achieve this goal, we propose a unified radiance field by introducing the reflection probability and tracing rays following the light transport model of Whitted Ray Tracing, and also develop several techniques to facilitate the learning process. Experiments and comparisons on both synthetic and real datasets demonstrate the superiority of our method. The code and supplementary material are available on the project webpage: https://zju3dv.github.io/Mirror-NeRF/.

23 October 2022
Verified
Book Chapter
Closed access
PDF file
YLYijin Li
Yijin Li
XLXinyang Liu
Xinyang Liu
WDWenqi Dong
Wenqi Dong
HZHan Zhou
Han Zhou
HBHujun Bao
Hujun Bao
ZCZhaopeng Cui
Zhaopeng Cui

16 March 2022 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
XDXiaoming Deng
Xiaoming Deng
DZDexin Zuo
Dexin Zuo
YZYinda Zhang
Yinda Zhang
ZCZhaopeng Cui
Zhaopeng Cui
JCJian Cheng
Jian Cheng
HWHongan Wang
Hongan Wang
3D hand pose estimation is a challenging problem in computer vision due to the high degrees-of-freedom of hand articulated motion space and large viewpoint variation. As a consequence, similar poses observed from multiple views can be dramatically different. In order to deal with this issue, view-independent features are required to achieve state-of-the-art performance. In this paper, we investigate the impact of view-independent features on 3D hand pose estimation from a single depth image, and propose a novel recurrent neural network for 3D hand pose estimation, in which a cascaded 3D pose-guided alignment strategy is designed for view-independent feature extraction and a recurrent hand pose module is designed for modeling the dependencies among sequential aligned features for 3D hand pose estimation. In particular, our cascaded pose-guided 3D alignments are performed in 3D space in a coarse-to-fine fashion. First, hand joints are predicted and globally transformed into a canonical reference frame; Second, the palm of the hand is detected and aligned; Third, local transformations are applied to the fingers to refine the final predictions. The proposed recurrent hand pose module for aligned 3D representation can extract recurrent pose-aware features and iteratively refines the estimated hand pose. Our recurrent module could be utilized for both single-view estimation and sequence-based estimation with 3D hand pose tracking. Experiments show that our method improves the state-of-the-art by a large margin on popular benchmarks with the simple yet efficient alignment and network architectures.

13 July 2024
Verified
Conference Paper
Open access
WDWenqi Dong
Wenqi Dong
BYBangbang Yang
Bangbang Yang
LMLin Ma
Lin Ma
XLXiao Liu
Xiao Liu
LCLiyuan Cui
Liyuan Cui
ZCZhaopeng Cui
Zhaopeng Cui
As humans, we aspire to create media content that is both freely willed and readily controlled. Thanks to the prominent development of generative techniques, we now can easily utilize 2D diffusion methods to synthesize images controlled by raw sketch or designated human poses, and even progressively edit/regenerate local regions with masked inpainting. However, similar workflows in 3D modeling tasks are still unavailable due to the lack of controllability and efficiency in 3D generation. In this paper, we present a novel controllable and interactive 3D assets modeling framework, named Coin3D. Coin3D allows users to control the 3D generation using a coarse geometry proxy assembled from basic shapes, and introduces an interactive generation workflow to support seamless local part editing while delivering responsive 3D object previewing within a few seconds. To this end, we develop several techniques, including the 3D adapter that applies volumetric coarse shape control to the diffusion model, proxy-bounded editing strategy for precise part editing, progressive volume cache to support responsive preview, and volume-SDS to ensure consistent mesh reconstruction. Extensive experiments of interactive generation and editing on diverse shape proxies demonstrate that our method achieves superior controllability and flexibility in the 3D assets generation task. Code and data are available on the project webpage: https://zju3dv.github.io/coin3d/.