Liuhao Ge

Liuhao Ge

Auto-generated profile, based on publicly available information

Liuhao Ge

Institute for Media Innovation, Nanyang Technological University, Singapore

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 October 2018
Verified
Book Chapter
Closed access
PDF file
LGLiuhao Ge
Liuhao Ge
ZRZhou Ren
Zhou Ren
JYJunsong Yuan
Junsong Yuan

16 April 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
LGLiuhao Ge
Liuhao Ge
HLHui Liang
Hui Liang
JYJunsong Yuan
Junsong Yuan
DTDaniel Thalmann
Daniel Thalmann
In this paper, we present a novel method for real-time 3D hand pose estimation from single depth images using 3D Convolutional Neural Networks (CNNs). Image-based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to the lack of 3D spatial information. Our proposed 3D CNN-based method, taking a 3D volumetric representation of the hand depth image as input and extracting 3D features from the volumetric input, can capture the 3D spatial structure of the hand and accurately regress full 3D hand pose in a single pass. In order to make the 3D CNN robust to variations in hand sizes and global orientations, we perform 3D data augmentation on the training data. To further improve the estimation accuracy, we propose applying the 3D deep network architectures and leveraging the complete hand surface as intermediate supervision for learning 3D hand pose from depth images. Extensive experiments on three challenging datasets demonstrate that our proposed approach outperforms baselines and state-of-the-art methods. A cross-dataset experiment also shows that our method has good generalization ability. Furthermore, our method is fast as our implementation runs at over 91 frames per second on a standard computer with a single GPU.

10 May 2018 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
LGLiuhao Ge
Liuhao Ge
HLHui Liang
Hui Liang
JYJunsong Yuan
Junsong Yuan
DTDaniel Thalmann
Daniel Thalmann
Articulated hand pose estimation is one of core technologies in human-computer interaction. Despite the recent progress, most existing methods still cannot achieve satisfactory performance, partly due to the difficulty of the embedded highdimensional nonlinear regression problem. Most existing datadriven methods directly regress 3D hand pose from 2D depth image, which cannot fully utilize the depth information. In this paper, we propose a novel multi-view CNN-based approach for 3D hand pose estimation. To better exploit 3D information in the depth image, we project the point cloud generated from the query depth image onto multiple views of two projection settings and integrate them for more robust estimation. Multi-view CNNs are trained to learn the mapping from projected images to heatmaps which reflect probability distributions of joints on each view. These multi-view heat-maps are then fused to estimate optimal 3D hand pose with learned pose priors, and unreliable information in multi-view heat-maps is suppressed using a view selection method. Experimental results show that the proposed method is superior to state-of-the-art methods on two challenging datasets. Furthermore, a cross-dataset experiment also validates that our proposed approach has good generalization ability.

11 May 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
YCYujun Cai
Yujun Cai
LGLiuhao Ge
Liuhao Ge
JCJianfei Cai
Jianfei Cai
NTNadia Magnenat Thalmann
Nadia Magnenat Thalmann
JYJunsong Yuan
Junsong Yuan
Compared with depth-based 3D hand pose estimation, it is more challenging to infer 3D hand pose from monocular RGB images, due to the substantial depth ambiguity and the difficulty of obtaining fully-annotated training data. Different from the existing learning-based monocular RGB-input approaches that require accurate 3D annotations for training, we propose to leverage the depth images that can be easily obtained from commodity RGB-D cameras during training, while during testing we take only RGB inputs for 3D joint predictions. In this way, we alleviate the burden of the costly 3D annotations in real-world dataset. Particularly, we propose a weakly-supervised method, adaptating from fully-annotated synthetic dataset to weakly-labeled real-world single RGB dataset with the aid of a depth regularizer, which serves as weak supervision for 3D pose prediction. To further exploit the physical structure of 3D hand pose, we present a novel CVAE-based statistical framework to embed the pose-specific subspace from RGB images, which can then be used to infer the 3D hand joint locations. Extensive experiments on benchmark datasets validate that our proposed approach outperforms baselines and state-of-the-art methods, which proves the effectiveness of the proposed depth regularizer and the CVAE-based framework.

22 December 2017 in IEEE Transactions on Cybernetics
Verified
Journal Article
Closed access
HLHui Liang
Hui Liang
JYJunsong Yuan
Junsong Yuan
JLJun Lee
Jun Lee
LGLiuhao Ge
Liuhao Ge
DTDaniel Thalmann
Daniel Thalmann
Vision-based hand pose estimation is important in human-computer interaction. While many recent works focus on full degree-of-freedom hand pose estimation, robust estimation of global hand pose remains a challenging problem. This paper presents a novel algorithm to optimize the leaf weights in a Hough forest to assist global hand pose estimation with a single depth camera. Different from traditional Hough forest, we propose to learn the vote weights stored at the leaf nodes of a forest in a principled way to minimize average pose prediction error, so that ambiguous votes are largely suppressed during prediction fusion. Experiments show that the proposed method largely improves pose estimation accuracy with optimized leaf weights on both synthesis and real datasets and performs favorably compared to state-of-the-art convolutional neural network-based methods. On real-world depth videos, the proposed method demonstrates improved robustness compared to several other recent hand tracking systems from both industry and academy. Moreover, we utilize the proposed method to build virtual/augmented reality applications to allow users to manipulate and examine virtual objects with bare hands.