Kangkan Wang

Kangkan Wang

Auto-generated profile, based on publicly available information

Kangkan Wang

School of Computer Science and Engineering, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, and Jiangsu Key Lab of Image and Video Understanding for Social Security, Nanjing University of Science and Technology, Nanjing, P.R. China

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 August 2022 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
KWKangkan Wang
Kangkan Wang
SPSida Peng
Sida Peng
XZXiaowei Zhou
Xiaowei Zhou
JYJian Yang
Jian Yang
GZGuofeng Zhang
Guofeng Zhang
This paper addresses the challenge of human performance capture from sparse multi-view or monocular videos. Given a template mesh of the performer, previous methods capture the human motion by non-rigidly registering the template mesh to images with 2D silhouettes or dense photometric alignment. However, the detailed surface deformation cannot be recovered from the silhouettes, while the photometric alignment suffers from instability caused by appearance variation in the videos. To solve these problems, we propose NerfCap, a novel performance capture method based on the dynamic neural radiance field (NeRF) representation of the performer. Specifically, a canonical NeRF is initialized from the template geometry and registered to the video frames by optimizing the deformation field and the appearance model of the canonical NeRF. To capture both large body motion and detailed surface deformation, NerfCap combines linear blend skinning with embedded graph deformation. In contrast to the mesh-based methods that suffer from fixed topology and texture, NerfCap is able to flexibly capture complex geometry and appearance variation across the videos, and synthesize more photo-realistic images. In addition, NerfCap can be pre-trained end to end in a self-supervised manner by matching the synthesized videos with the input videos. Experimental results on various datasets show that NerfCap outperforms prior works in terms of both surface reconstruction accuracy and novel-view synthesis quality.

26 February 2020 in The Visual Computer
Verified
Journal Article
Closed access
PDF file
KWKangkan Wang
Kangkan Wang
GZGuofeng Zhang
Guofeng Zhang
JYJian Yang
Jian Yang
HBHujun Bao
Hujun Bao

7 January 2022 in The Visual Computer
Verified
Journal Article
Closed access
PDF file
KWKangkan Wang
Kangkan Wang
GZGuofeng Zhang
Guofeng Zhang
JYJian Yang
Jian Yang

6 October 2021 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
KWKangkan Wang
Kangkan Wang
GZGuofeng Zhang
Guofeng Zhang
HZHuayu Zheng
Huayu Zheng
JYJian Yang
Jian Yang
We propose a novel deep learning method to predict dense correspondences for partial point clouds of non-rigidly deformable targets. Dense correspondences are learned in the form of vertex displacements of a template mesh towards the point clouds. A two-stage regression framework is proposed to estimate accurate displacement vectors, including the global and local regression networks. Specifically, the global regression network estimates global displacements from the global features of the template mesh and point clouds through a graph CNN based hierarchical encoder-decoder network. Based on the initial displacements, a mesh can be generated that fits to the point clouds roughly. In the local regression network, a local feature embedding layer fuses local features of point clouds with graph features on the generated mesh through an attention mechanism. Consequently, the embedded local features are employed to refine the correspondences in local regions of the targets by predicting the increments of vertex displacements. Our method is further generalized to correspondence estimation on unseen real data with a robust fine-tuning method. The experimental results on diverse datasets of various deformable subjects (e.g., human bodies, animals, and hands) demonstrate that the proposed approach can accurately and robustly estimate dense correspondences from non-rigid point clouds.

30 July 2025 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
KWKangkan Wang
Kangkan Wang
CWChong Wang
Chong Wang
JYJian Yang
Jian Yang
GZGuofeng Zhang
Guofeng Zhang
Capturing the human body and clothing from videos has obtained significant progress in recent years, but several challenges remain to be addressed. Previous methods reconstruct the 3D bodies and garments from videos with self-rotating human motions or capture the body and clothing separately based on neural implicit fields. However, the reconstruction methods for self-rotating motions may cause instable tracking on dynamic videos with arbitrary human motions, while implicit fields based methods are limited to inefficient rendering and low quality synthesis. To solve these problems, we propose a new method, called CloCap-GS, for clothed human performance capture with 3D Gaussian Splatting. Specifically, we align 3D Gaussians with the deforming geometries of body and clothing, and leverage photometric constraints formed by matching Gaussians renderings with input video frames to recover temporal deformations of the dense template geometry. The geometry deformations and Gaussians properties of both the body and clothing are optimized jointly, achieving both dense geometry tracking and novel-view synthesis. In addition, we introduce a physics-aware material-varying cloth model to preserve physically-plausible cloth dynamics and body-clothing interactions that is pre-trained in a self-supervised manner without preparing training data. Compared with the existing methods, our method improves the accuracy of dense geometry tracking and quality of novel-view synthesis for a variety of daily garment types (e.g., loose clothes). Extensive experiments in both quantitative and qualitative evaluations demonstrate the effectiveness of CloCap-GS on real sparse-view or monocular videos.

11 October 2020
Verified
Book Chapter
Closed access
PDF file
LLLei Liu
Lei Liu
KWKangkan Wang
Kangkan Wang
JYJian Yang
Jian Yang