Jiangjing Lyu

Jiangjing Lyu

Auto-generated profile, based on publicly available information

Jiangjing Lyu

Alibaba Group

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 March 2017 in Neurocomputing
Verified
Journal Article
Closed access
JLJiang-Jing Lv
Jiang-Jing Lv
XSXiao-Hu Shao
Xiao-Hu Shao
JHJia-Shui Huang
Jia-Shui Huang
XZXiang-Dong Zhou
Xiang-Dong Zhou
XZXi Zhou
Xi Zhou

16 May 2023 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
YKYueying Kao
Yueying Kao
BPBowen Pan
Bowen Pan
MXMiao Xu
Miao Xu
JLJiangjing Lyu
Jiangjing Lyu
XZXiangyu Zhu
Xiangyu Zhu
ZLZhen Lei
Zhen Lei
In 3D face reconstruction, orthogonal projection has been widely employed to substitute perspective projection to simplify the fitting process. This approximation performs well when the distance between camera and face is far enough. However, in some scenarios that the face is very close to camera or moving along the camera axis, the methods suffer from the inaccurate reconstruction and unstable temporal fitting due to the distortion under the perspective projection. In this paper, we aim to address the problem of single-image 3D face reconstruction under perspective projection. Specifically, a deep neural network, Perspective Network (PerspNet), is proposed to simultaneously reconstruct 3D face shape in canonical space and learn the correspondence between 2D pixels and 3D points, by which the 6DoF (6 Degrees of Freedom) face pose can be estimated to represent perspective projection. Besides, we contribute a large ARKitFace dataset to enable the training and evaluation of 3D face reconstruction solutions under the scenarios of perspective projection, which has 902,724 2D facial images with ground-truth 3D face mesh and annotated 6DoF pose parameters. Experimental results show that our approach outperforms current state-of-the-art methods by a significant margin. The code and data are available at https://github.com/cbsropenproject/6dof_face.

3 March 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
SLShuichang Lai
Shuichang Lai
LHLetian Huang
Letian Huang
JGJie Guo
Jie Guo
KCKai Cheng
Kai Cheng
BPBowen Pan
Bowen Pan
YGYanwen Guo
Yanwen Guo
Reconstructing objects from posed images is a crucial and complex task in computer graphics and computer vision. While NeRF-based neural reconstruction methods have exhibited impressive reconstruction ability, they tend to be time-comsuming. Recent strategies have adopted 3D Gaussian Splatting (3D-GS) for inverse rendering, which have led to quick and effective outcomes. However, these techniques generally have difficulty in producing believable geometries and materials for glossy objects, a challenge that stems from the inherent ambiguities of inverse rendering. To address this, we introduce GlossyGS, an innovative 3D-GS-based inverse rendering framework that aims to precisely reconstruct the geometry and materials of glossy objects by integrating material priors. The key idea is the use of micro-facet geometry segmentation prior, which helps to reduce the intrinsic ambiguities and improve the decomposition of geometries and materials. Additionally, we introduce a normal map prefiltering strategy to more accurately simulate the normal distribution of reflective surfaces. These strategies are integrated into a hybrid geometry and material representation that employs both explicit and implicit methods to depict glossy objects. We demonstrate through quantitative analysis and qualitative visualization that the proposed method is effective to reconstruct high-fidelity geometries and materials of glossy objects, and performs favorably against State-of-the-Arts.

16 May 2023 in IEEE Transactions on Biometrics, Behavior, and Identity Science
Verified
Journal Article
Closed access
XZXiangyu Zhu
Xiangyu Zhu
TLTingting Liao
Tingting Liao
XZXiaomei Zhang
Xiaomei Zhang
JLJiangjing Lyu
Jiangjing Lyu
ZCZhiwen Chen
Zhiwen Chen
ZLZhen Lei
Zhen Lei
In this paper, we consider a novel problem of reconstructing a 3D clothed human avatar from multiple frames, independent of assumptions on camera calibration, capture space, and constrained actions. We contribute a large-scale dataset, Multi-View and multi-Pose 3D human (MVP-Human in short) to help address this problem. The dataset contains 400 subjects, each of which has 15 scans in different poses and 8-view images for each pose, providing 6,000 3D scans and 48,000 images in total. In addition, a baseline method that takes multiple images as inputs, and generates a shape-with-skinning avatar in the canonical space, finished in one feed-forward pass is proposed. It first reconstructs the implicit skinning fields in a multi-level manner, and then the image features from multiple images are aligned and integrated to estimate a pixel-aligned implicit function that represents the clothed shape. With the newly collected dataset and the baseline method, it shows promising performance on 3D clothed avatar reconstruction. We release the MVP-Human dataset and the baseline method in, hoping to promote research and development in this field.

15 April 2021 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
XSXiaohu Shao
Xiaohu Shao
JXJunliang Xing
Junliang Xing
JLJiangjing Lyu
Jiangjing Lyu
XZXiangdong Zhou
Xiangdong Zhou
YSYu Shi
Yu Shi
SMStephen Maybank
Stephen Maybank
Regression-based face alignment involves learning a series of mapping functions to predict the true landmarks from an initial estimation of the alignment. Most existing approaches focus on learning efficacious mapping functions from some feature representations to improve performance. The issues related to the initial alignment estimation and the final learning objective, however, receive less attention. This work proposes a deep regression architecture with progressive reinitialization and a new error-driven learning loss function to explicitly address the above two issues. Given an image with a rough face detection result, the full face region is first mapped by a supervised spatial transformer network to a normalized form and trained to regress coarse positions of landmarks. Then, different face parts are further respectively reinitialized to their own normalized states, followed by another regression sub-network to refine the landmark positions. To deal with the inconsistent annotations in existing training datasets, we further propose an adaptive landmark-weighted loss function. It dynamically adjusts the importance of different landmarks according to their learning errors during training without depending on any hyper-parameters manually set by trial and error. A high level of robustness to annotation inconsistencies is thus achieved. The whole deep architecture permits training from end to end, and extensive experimental analyses and comparisons demonstrate its effectiveness and efficiency. The source code, trained models, and experimental results are made available at https://github.com/shaoxiaohu/Face_Alignment_DPR.git .

1 October 2019
Verified
Conference Paper
Closed access
XSXiaohu Shao
Xiaohu Shao
JLJiangjing Lyu
Jiangjing Lyu
JXJunliang Xing
Junliang Xing
LZLijun Zhang
Lijun Zhang
XLXiaobo Li
Xiaobo Li
YSYu Shi
Yu Shi
This paper introduces our submission to the 2nd 3DFAW Challenge. To get a high-accuracy 3D dense face shape based on 2D videos or multiple images, a framework which is consist of multi-reconstruction branches and a mesh retrieval module, is proposed to effectively utilize the information of all frames and the results predicted by all branches. The recent state-of-the-art methods based on single-view and multi-view are introduced to form an ensemble of independent regression networks. The candidate 3D shape of each branch is synthesized by weighted linear combination of the results on all frames to boost the depth estimation and invisible regions reconstruction. Finally, the best fitting mesh is retrieved according to the distance between the synthesized texture and the ground truth texture. Experiment results show that our approach obtains competitive results near the accuracy of "pseudo" ground truths, and achieves superior performance over most of submissions by other teams in the testing phases.