Lizhi Zhao

Lizhi Zhao

Auto-generated profile, based on publicly available information

Lizhi Zhao

State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 September 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JJJincen Jiang
Jincen Jiang
XLXuequan Lu
Xuequan Lu
LZLizhi Zhao
Lizhi Zhao
RDRichard Dazeley
Richard Dazeley
MWMEILI WANG
MEILI WANG
Transformer-based Self-supervised Representation Learning methods learn generic features from unlabeled datasets for providing useful network initialization parameters for downstream tasks. Recently, methods based upon masking Autoencoders have been explored in the fields. The input can be intuitively masked due to regular content, like sequence words and 2D pixels. However, the extension to 3D point cloud is challenging due to irregularity. In this paper, we propose masked Autoencoders in 3D point cloud representation learning (abbreviated as MAE3D), a novel autoencoding paradigm for self-supervised learning. We first split the input point cloud into patches and mask a portion of them, then use our Patch Embedding Module to extract the features of unmasked patches. Secondly, we employ patch-wise MAE3D Transformers to learn both local features of point cloud patches and high-level contextual relationships between patches, then complete the latent representations of masked patches. We use our Point Cloud Reconstruction Module with multi-task loss to complete the incomplete point cloud as a result. We conduct self-supervised pre-training on ShapeNet55 with the point cloud completion pre-text task and fine-tune the pre-trained model on ModelNet40 and ScanObjectNN (PB_T50_RS, the hardest variant). Comprehensive experiments demonstrate that the local features extracted by our MAE3D from point cloud patches are beneficial for downstream classification tasks, soundly outperforming state-of-the-art methods (93.4% and 86.2% classification accuracy, respectively). Our source codes are available at: https://github.com/Jinec98/MAE3D.

18 March 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
RFRunze Fan
Runze Fan
JWJian Wu
Jian Wu
XSXuehuai Shi
Xuehuai Shi
LZLizhi Zhao
Lizhi Zhao
Qixiang MaQixiang Ma
Qixiang Ma
LWLili Wang
Lili Wang
Rendering quality and performance greatly affect the user's immersion in VR experiences. 3D Gaussian Splatting-based methods can achieve photo-realistic rendering with speeds of over 100 fps in static scenes, but the speed drops below 10 fps in monocular dynamic scenes. Foveated rendering provides a possible solution to accelerate rendering without compromising visual perceptual quality. However, 3DGS and foveated rendering are not compatible. In this paper, we propose Fov-GS, a foveated 3D Gaussian splatting method for rendering dynamic scenes in real time. We introduce a 3D Gaussian forest representation that represents the scene as a forest. To construct the 3D Gaussian forest, we propose a 3D Gaussian forest initialization method based on dynamic-static separation. Subsequently, we propose a 3D Gaussian forest optimization method based on deformation field and Gaussian decomposition to optimize the forest and deformation field. To achieve real-time dynamic scene rendering, we present a 3D Gaussian forest rendering method based on HVS models. Experiments demonstrate that our method not only achieves higher rendering quality in the foveal and salient regions compared to the SOTA methods but also dramatically improves rendering performance, achieving up to 11.33X speedup. We also conducted a user study, and the results prove that the perceptual quality of our method has a high visual similarity with the ground truth.

13 December 2024 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
LZLizhi Zhao
Lizhi Zhao
XLXuequan Lu
Xuequan Lu
RFRunze Fan
Runze Fan
SISio-Kei Im
Sio-Kei Im
LWLili Wang
Lili Wang
Rendering animatable and realistic hand avatars is pivotal for enhancing user experiences in human-centered AR/VR applications. While recent initiatives have utilized neural radiance fields to forge hand avatars with lifelike appearances, these methods are often hindered by high computational demands and the necessity for extensive training views. In this paper, we introduce GaussianHand, the first Gaussian-based real-time 3D rendering approach that enables efficient free-view and free-pose hand avatar animation from sparse view images. Our approach encompasses two key innovations. We first propose Hand Gaussian Blend Shapes that effectively models hand surface geometry while ensuring consistent appearance across various poses. Second, we introduce the Neural Residual Skeleton, equipped with Residual Skinning Weights, designed to rectify inaccuracies involved in Linear Blend Skinning deformations due to geometry offsets. Experiments demonstrate that our method not only achieves far more realistic rendering quality with as few as 5 or 20 training views, compared to the 139 views required by existing methods, but also excels in efficiency, achieving up to 125 frames per second for real-time rendering and remarkably surpassing recent methods.

27 December 2022
Verified
Conference Paper
Open access
YSYinghan Shi
Yinghan Shi
LZLizhi Zhao
Lizhi Zhao
XLXuequan Lu
Xuequan Lu
THThuong Hoang
Thuong Hoang
MWMEILI WANG
MEILI WANG
Virtual hands are typically hidden after grasping an object in Virtual Reality (VR) environments in existing virtual hand grasping works, which significantly impact user experience. In this paper, we build a real-time, flexible, robust and natural virtual hand grasping system, in which the virtual hand is an avatar of the user’s real hand, and a user can control the virtual hand to grasp different rigid objects naturally and realistically in the VR environment by grasping the controller handle. Our method involves three modules: Grasping Detection module for detecting whether there is a graspable object, Hand-Object Connection module for attaching the graspable object to the virtual hand, and Finger Bending module for controlling fingers to bend to grasp objects. We conduct experiments on using virtual hand to grasp rigid objects and manipulate physical tools, and show that the virtual hand can fit the object very well. Also, we compare our method with the above hand hidden technique using two created VR scenarios, which demonstrates the superiority of our method.

2 October 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
Qixiang MaQixiang Ma
Qixiang Ma
Runze FanRunze Fan
Runze Fan
LZLizhi Zhao
Lizhi Zhao
JWJian Wu
Jian Wu
SISio-Kei Im
Sio-Kei Im
LWLili Wang
Lili Wang
3D scene graph generation is essential for spatial computing in Extended Reality (XR), providing structured semantics for task planning and intelligent perception. However, unlike instance-segmentation-driven setups, generating semantic scene graphs still suffer from limited accuracy due to coarse and noisy point cloud data typically acquired in practice, and from the lack of interactive strategies to incorporate users, spatialized and intuitive guidance. We identify three key challenges: designing controllable interaction forms, involving guidance in inference, and generalizing from local corrections. To address these, we propose SGSG, a Stroke-Guided Scene Graph generation method that enables users to interactively refine 3D semantic relationships and improve predictions in real time. We propose three types of strokes and a lightweight SGstrokes dataset tailored for this modality. Our model integrates stroke guidance representation and injection for spatio-temporal feature learning and reasoning correction, along with intervention losses that combine consistency-repulsive and geometry-sensitive constraints to enhance accuracy and generalization. Experiments and the user study show that SGSG outperforms state-of-the-art methods 3DSSG and SGFN in overall accuracy and precision, surpasses JointSSG in predicate-level metrics, and reduces task load across all control conditions, establishing SGSG as a new benchmark for interactive 3D scene graph generation and semantic understanding in XR. Implementation resources are available at: https://github.com/Sycamore-Ma/SGSG-runtime.