Jinsheng Wei

Jinsheng Wei

Auto-generated profile, based on publicly available information

Jinsheng Wei

Jiangsu Key Laboratory of Intelligent Information Processing and Communication Technology, College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 October 2022 in IEEE Transactions on Affective Computing
Verified
Journal Article
Open access
YLYante Li
Yante Li
JWJinsheng Wei
Jinsheng Wei
YLYang Liu
Yang Liu
JKJanne Kauttonen
Janne Kauttonen
GZGuoying Zhao
Guoying Zhao
Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

6 December 2023 in IEEE Transactions on Affective Computing
Verified
Journal Article
Closed access
JWJinsheng Wei
Jinsheng Wei
WPWei Peng
Wei Peng
GLGuanming Lu
Guanming Lu
YLYante Li
Yante Li
JYJing-Jie Yan
Jing-Jie Yan
GZGuoying Zhao
Guoying Zhao
Micro-expression recognition (MER) holds significance in uncovering hidden emotions. Most works take image sequences as input and cannot effectively explore ME information because subtle ME-related motions are easily submerged in unrelated information. Instead, the facial landmark is a lowdimensional and compact modality, which achieves lower computational cost and potentially concentrates on ME-related movement features. However, the discriminability of facial landmarks for MER is unclear. Thus, this paper investigates the contribution of facial landmarks and proposes a novel framework to efficiently recognize MEs with facial landmarks. Firstly, a geometric twostream graph network is constructed to aggregate the low-order and high-order geometric movement information from facial landmarks to obtain discriminative ME representation. Secondly, a self-learning fashion is introduced to automatically model the dynamic relationship between nodes even long-distance nodes. Furthermore, an adaptive action unit loss is proposed to reasonably build a strong correlation between landmarks, facial action units and MEs. Notably, this work provides a novel idea with much higher efficiency to promote MER, only utilizing graphbased geometric features. The experimental results demonstrate that the proposed method achieves competitive performance with a significantly reduced computational cost. Furthermore, facial landmarks significantly contribute to MER and are worth further study for high-efficient ME analysis.

21 September 2024 in Electronics
Verified
Journal Article
Open access
PDF file
JYJing-Jie Yan
Jing-Jie Yan
PLPeiyuan Li
Peiyuan Li
CDChengkun Du
Chengkun Du
KZKang Zhu
Kang Zhu
XZXiaoyang Zhou
Xiaoyang Zhou
JWJinsheng Wei
Jinsheng Wei
The research of multimodal emotion recognition based on facial expressions, speech, and body gestures is crucial for oncoming intelligent human–computer interfaces. However, it is a very difficult task and has seldom been researched in this combination in the past years. Based on the GEMEP and Polish databases, this contribution focuses on trimodal emotion recognition from facial expressions, speech, and body gestures, including feature extraction, feature fusion, and multimodal classification of the three modalities. In particular, for feature fusion, two novel algorithms including supervised least squares multiset kernel canonical correlation analysis (SLSMKCCA) and sparse supervised least squares multiset kernel canonical correlation analysis (SSLSMKCCA) are presented, respectively, to carry out efficient facial expression, speech, and body gesture feature fusion. Different from the traditional multiset kernel canonical correlation analysis (MKCCA) algorithms, our SLSKMCCA algorithm is a supervised version and is based on the least squares form. The SSLSKMCCA algorithm is implemented by the combination of SLSMKCCA and a sparse item (L1 Norm). Moreover, two effective solving algorithms for SLSMKCCA and SSLSMKCCA are presented in addition, which use the alternated least squares and augmented Lagrangian multiplier methods, respectively. The extensive experimental results on the popular public GEMEP and Polish databases show that the recognition rate of multimodal emotion recognition is superior to bimodal and monomodal emotion recognition on average, and our presented SLSMKCCA and SSLSMKCCA fusion methods both obtain very high recognition rates, especially for the SSLSMKCCA fusion method.

27 May 2019 in IEEE Access
Verified
Journal Article
Open access
JWJinsheng Wei
Jinsheng Wei
GLGuanming Lu
Guanming Lu
HLHuaming Liu
Huaming Liu
JYJingjie Yan
Jingjie Yan
Facial image inpainting is a challenging task because the missing region needs to be filled by the new pixels with semantic information (e.g., noses and mouths). The traditional methods that involve searching for similar patches are mature but it is not suitable for semantic inpainting. Recently, the deep generative model-based methods have been able to implement semantic image inpainting although inpainting results are blurry or distorted. In this paper, through analyzing the advantages and disadvantages of the two methods, we propose a novel and efficient method that combines these two methods by a series connection, which searches for the most reasonable similar patch using the coarse image generated by the deep generative model. When training model, adding Laplace Loss to standard loss accelerates model convergence. In addition, we define Region Weight (RW) when searching for similar patches, which makes edge connection more natural. Our method addresses the problem of blurred results in the deep generative model and dissatisfactory semantic information in the traditional methods. Our experiments, which used the CelebA dataset, demonstrate that our method can achieve realistic and natural facial inpainting results.

27 May 2023 in Multimedia Tools and Applications
Verified
Journal Article
Closed access
PDF file
GLGuanming Lu
Guanming Lu
HCHaoxia Chen
Haoxia Chen
JWJinsheng Wei
Jinsheng Wei
XLxiaonan Li
xiaonan Li
XZXianlan Zheng
Xianlan Zheng
JYJing-Jie Yan
Jing-Jie Yan

10 June 2023 in Multimedia Tools and Applications
Verified
Journal Article
Closed access
PDF file
GLGuanming Lu
Guanming Lu
ZHZhen Han
Zhen Han
JWJinsheng Wei
Jinsheng Wei
JYJing-Jie Yan
Jing-Jie Yan