Shufei Duan

Dr. Shufei Duan

orcid

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 September 2022 in Applied Sciences
Verified
Journal Article
Open access
PDF file
JLJuan Li
Juan Li
XZXueying Zhang
Xueying Zhang
LHLixia Huang
Lixia Huang
FLFenglian Li
Fenglian Li
SDShufei Duan
Shufei Duan
YSYing Sun
Ying Sun
In the background of artificial intelligence, the realization of smooth communication between people and machines has become the goal pursued by people. Mel spectrograms is a common method used in speech emotion recognition, focusing on the low-frequency part of speech. In contrast, the inverse Mel (IMel) spectrogram, which focuses on the high-frequency part, is proposed to comprehensively analyze emotions. Because the convolutional neural network-stacked sparse autoencoder (CNN-SSAE) can extract deep optimized features, the Mel-IMel dual-channel complementary structure is proposed. In the first channel, a CNN is used to extract the low-frequency information of the Mel spectrogram. The other channel extracts the high-frequency information of the IMel spectrogram. This information is transmitted into an SSAE to reduce the number of dimensions, and obtain the optimized information. Experimental results show that the highest recognition rates achieved on the EMO-DB, SAVEE, and RAVDESS datasets were 94.79%, 88.96%, and 83.18%, respectively. The conclusions are that the recognition rate of the two spectrograms was higher than that of each of the single spectrograms, which proves that the two spectrograms are complementary. The SSAE followed the CNN to get the optimized information, and the recognition rate was further improved, which proves the effectiveness of the CNN-SSAE network.

1 November 2015
Verified
Conference Paper
Closed access
XZXueying Zhang
Xueying Zhang
YSYing Sun
Ying Sun
SDShufei Duan
Shufei Duan
Emotional information in speech signal is an important information resource. When verbal expression combined with human emotion, emotional speech processing is no longer a simple mathematical model or pure calculations. Fluctuations of the mood are controlled by the brain perception; speech signal processing based on cognitive psychology can capture emotion better. In this paper the relevance analysis between speech emotion and human cognition is introduced firstly. The recent progress in speech emotion recognition was summarized including the review of speech emotion databases, feature extraction and emotion recognition networks. Secondly a fuzzy cognitive map network based on cognitive psychology is introduced into emotional speech recognition. In addition, the mechanism of the human brain for cognitive emotional speech is explored. To improve the recognition accuracy, this report also tries to integrate event-related potentials to speech emotion recognition. This idea is the conception and prospect of speech emotion recognition mashed up with cognitive psychology in the future.

11 May 2020 in IEEE Access
Verified
Journal Article
Open access
SDShufei Duan
Shufei Duan
XZXueying Zhang
Xueying Zhang
MYMimi Yan
Mimi Yan
Jinglan ZhangJinglan Zhang
Jinglan Zhang
Pathological articulation exploration, especially the study of the kinematic characteristics of motor organ, is helpful to further reveal the essence of motor dysarthria. Due to the scarcity of the available pathological pronunciation database, there has little research working on the statistic distribution analysis for patients and normal controlled people. This paper applied the distribution method on TORGO database to discover the cognitive and motor rules of dysarthria patients. Single phoneme analysis is effective for locating the specific tongue muscle but ignoring cognitive ability assessment, particularly for the content understanding and the fluency degree of expression by patients. The paper focused on the word/sentence level rather than single phoneme analysis. The reaction time was designed to reveal the relationship between the brain cognition and motor neuron activation. The statistic distribution tells that the cerebral palsy or amyotrophic lateral sclerosis does affect people’s reflection and make the patients hard to control the tongue muscles effectively, resulting in unstable reaction time. The articulation velocity of patients appears 5mm/s faster than normal people, at 85mm/s, perhaps due to the factors of the word/sentence data and the big proportion of extra large displacement. It illustrates that the tongue moves relatively coherently and fluently once patients active the muscles, but hard to slow down as the muscle control ability decreases. The spatial occupancy was represented by the maximum articulation movement range (MAMR). We adapted the logarithmic normal distribution to find out the significant threshold for the diagnosis of dysarthria with the MAMR exceeding 7mm along left and right direction and the number of abnormal ranges surpassing 10% of total number. Primary test of MAMR, as an articulatory feature, for speech classification was carried out and it achieves 81% accuracy. These explorations convinced us to apply them to the pathological speech recognition task for improvement in future.

15 January 2025 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Verified
Journal Article
Open access
TZTing Zhu
Ting Zhu
SDShufei Duan
Shufei Duan
HLHuizhi Liang
Huizhi Liang
FLFujiang Li
Fujiang Li
WZWei Zhang
Wei Zhang
The clinical diagnosis and treatment of motor dysarthria in post-stroke patients is often subjective and neglects the impact of psychological and emotional disorders on disease progression. This study aims to analyze the correlation among emotional expression, psychological state, facial expression, and dysarthria disease severity and is dedicated to the construction of a dysarthria prediction model. We first designed THE-POSSD, a novel Chinese multimodal emotional pathology expression database, which collected acoustic, glottal, and facial data under emotional stimuli from patients at different disease stages and healthy controls. Emotional speech was labeled for intelligibility scores, emotion types, and discrete dimensional space. Then, their correlation with disease development was investigated and analyzed. A total of 154 significant correlation features were extracted for analysis. To mitigate the limitations of subjective clinical scale diagnosis and account for psychological and emotional factors, this study introduced the grey correlation theory and constructed a dysarthria prediction model based on the grey relational analysis-deep belief network (GRA-DBN). Principal Component Analysis and Variance Inflation Factor were employed to optimize GRA-DBN model. Both proposed models achieved a high prediction accuracy, with an adjusted R² value of 0.85 for GRA-DBN and 0.92 for optimised model. This study fills the gap in the international multimodal emotional pathological expression dataset and provides a comprehensive framework for analyzing the association between mental state, emotional expression, and the degree of dysarthria. Furthermore, the incorporation of key multimodal features into the predictive model highlights its potential to enhance the precision of clinical diagnostic processes significantly.

16 July 2024 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
XQXiaoyi Qin
Xiaoyi Qin
NLNa Li
Na Li
SDShufei Duan
Shufei Duan
MLMing Li
Ming Li
The performance of speaker verification systems can be adversely affected by time domain variations. However, limited research has been conducted on time-varying speaker verification due to the absence of appropriate datasets. This paper aims to investigate the impact of long-term and short-term time-varying in speaker verification and proposes solutions to mitigate these effects. For long-term speaker verification (i.e., cross-age speaker verification), we introduce an age-decoupling adversarial learning method to learn age-invariant speaker representation by mining age information from the VoxCeleb dataset. For short-term speaker verification, we collect the SMIIP-TimeVarying (SMIIP-TV) Dataset, which includes recordings at multiple time slots every day from 373 speakers for 90 consecutive days and other relevant meta information. Using this dataset, we analyze the time-varying of speaker embeddings and propose a novel but realistic time-varying speaker verification task, termed incremental sequence-pair speaker verification. This task involves continuous interaction between enrollment audios and a sequence of testing audios with the aim of improving performance over time. We introduce the template updating method to counter the negative effects over time, and then formulate the template updating processing as a Markov Decision Process and propose a template updating method based on deep reinforcement learning (DRL). The policy network of DRL is treated as an agent to determine if and how much should the template be updated. In summary, this paper releases our collected database, investigates both the long-term and short-term time-varying scenarios and provides insights and solutions into time-varying speaker verification.

13 February 2026 in Sensors
Verified
Journal Article
Open access
PDF file
SDShufei Duan
Shufei Duan
YGYuxin Guo
Yuxin Guo
LFLonghao Fu
Longhao Fu
FLFujiang Li
Fujiang Li
XDXinran Dong
Xinran Dong
WZWei Zhang
Wei Zhang
Dysarthria in patients post-stroke is often accompanied by central facial paralysis, which impairs facial motor control and emotional expression. Current assessments rely on acoustic modalities, overlooking facial pathological cues and their correlation with emotional expression, which hinders comprehensive disease assessment. To address this issue, we propose a multimodal severity classification framework that integrates facial and acoustic features. Firstly, a multi-level annotation algorithm based on a pre-trained model and motion amplitude was designed to overcome the problem of data scarcity. Secondly, facial topology was modeled using Delaunay triangulation, with spatial relationships captured via graph convolutional networks (GCNs), while abnormal muscle coordination is quantified using facial action units (AUs). Finally, we proposed a multimodal feature set fusion technology framework to achieve the compensation of facial visual features for acoustic modalities and the analysis of disease classification. Our experimental results using the THE-POSSD dataset demonstrate an accuracy of 92.0% and an F1 score of 91.6%, significantly outperforming single-modality baselines. This study reveals the changes in facial movements and sensitive areas of patients under different emotional states, verifies the compensatory ability of visual patterns for auditory patterns, and demonstrates the potential of this multimodal framework for objective assessment and future clinical applications in speech disorders.