Aishan Wumaier

Prof. Dr. Aishan Wumaier

School of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
orcid

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 June 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
JLJiajun Liu
Jiajun Liu
AWAishan Wumaier
Aishan Wumaier
DWDongPing Wei
DongPing Wei
SGshen Mr. guo
shen Mr. guo
Speech is critical for interpersonal communication, but not everyone has fluent communication skills. Speech disfluency, including stuttering and interruptions, affects not only emotional expression but also clarity of expression for people who stutter. Existing methods for detecting speech disfluency rely heavily on annotated data, which can be costly. Additionally, these methods have not considered the issue of variable-length disfluent speech, which limits the scalability of detection methods. To address these limitations, this paper proposes an automated method for detecting speech disfluency that can improve communication skills for individuals and assist therapists in tracking the progress of stuttering patients. The proposed method focuses on detecting four types of disfluency features using single-task detection and utilizes embeddings from the pre-trained wav2vec2.0 model, as well as convolutional neural network (CNN) and Transformer models for feature extraction. The model’s scalability is improved by considering the issue of variable-length disfluent speech and modifying the model based on the entropy invariance of attention mechanisms. The proposed automated method for detecting speech disfluency has the potential to assist individuals in overcoming speech disfluency, improve their communication skills, and aid therapists in tracking the progress of stuttering patients. Additionally, the model’s scalability across languages and lengths enhances its practical applicability. The experiments demonstrate that the model outperforms baseline models in both English and Chinese datasets, proving its universality and scalability in real-world applications.

19 April 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
XXXiangzhe Xin
Xiangzhe Xin
AWAishan Wumaier
Aishan Wumaier
ZKZaokere Kadeer
Zaokere Kadeer
JHjangtao he
jangtao he
Aspect-level sentiment analysis aims to identify the sentiment polarity of specific aspects appearing in a given sentence or review. The model based on graph structure uses a dependency tree to link the aspect word with its corresponding opinion word and achieves significant results. However, for some sentences with ambiguous syntactic structure, it is difficult for the dependency tree to accurately parse the dependencies, which introduces noise and degrades the performance of the model. Based on this, we propose a syntactic and semantic enhanced multi-layer graph attention network (SSEMGAT), which introduces constituent trees in syntactic features to compensate for dependent trees at the clause level, exploiting aspect-aware attention in semantic features to assign the attention weight of specific aspects between contexts. The enhanced syntactic and semantic features are then used to classify specific aspects of sentiment through a multi-layer graph attention network. Accuracy and Macro-F1 are used as evaluation indexes in the SemEval-2014 Task 4 Restaurant and Laptop dataset and the Twitter dataset to compare the proposed model with the baseline model and the latest model, achieving competitive results.

12 December 2022 in IEEE Access
Verified
Journal Article
Open access
QYQian Yang
Qian Yang
ZKZaokere Kadeer
Zaokere Kadeer
WGWenxia Gu
Wenxia Gu
WSWeiwei Sun
Weiwei Sun
AWAishan Wumaier
Aishan Wumaier
Aspect-based sentiment analysis(ABSA) aims to identify the sentiment polarity of specific aspects in sentences, which can more accurately mine the sentiment polarity of users towards different aspects. Most of the existing works derive the sentiment features of specific aspects by interactively learning the dependencies between different aspects of the context. However, the above work has neglected to use the external affective commonsense knowledge to augment the ability of the Graph Convolutional Networks(GCNs) to interactively capture sentiment dependencies of the inter-aspect words in different contexts. In addition, compared to the ABSA research in English, the existing research pays less attention to the Chinese-oriented research. Meanwhile, multi-head self-sttention(MHSA) is applied to extract richer context syntax and semantic interaction features. In this paper, we propose a novel knowledge-aware model in which affective knowledge augments interactive GCN for Chinese-oriented ABSA, namely AKM-IGCN. Moreover, this model can be applied to effectively analyze both Chinese and English comments simultaneously. Hence, we conducted experiments on four Chinese datasets(Camera, Phone, Notebook and Car) and six English benchmark datasets(Restaurant14, Restaurant15, Restaurant16, Twitter, MAMS, Tshirt). Experimental results illustrate that our proposed model outperforms or approaches state-of-the-art models.

9 April 2023 in Electronics
Verified
Journal Article
Open access
PDF file
JLJiajun Liu
Jiajun Liu
AWAishan Wumaier
Aishan Wumaier
CFCong Fan
Cong Fan
SGshen Mr. guo
shen Mr. guo
The automatic fluency assessment of spontaneous speech without reference text is a challenging task that heavily depends on the accuracy of automatic speech recognition (ASR). Considering this scenario, it is necessary to explore an assessment method that combines ASR. This is mainly due to the fact that in addition to acoustic features being essential for assessment, the text features output by ASR may also contain potentially fluency information. However, most existing studies on automatic fluency assessment of spontaneous speech are based solely on audio features, without utilizing textual information, which may lead to a limited understanding of fluency features. To address this, we propose a multimodal automatic speech fluency assessment method that combines ASR output. Specifically, we first explore the relevance of the fluency assessment task to the ASR task and fine-tune the Wav2Vec2.0 model using multi-task learning to jointly optimize the ASR task and fluency assessment task, resulting in both the fluency assessment results and the ASR output. Then, the text features and audio features obtained from the fine-tuned model are fed into the multimodal fluency assessment model, using attention mechanisms to obtain more reliable assessment results. Finally, experiments on the PSCPSF and Speechocean762 dataset suggest that our proposed method performs well in different assessment scenarios.

14 November 2023 in Speech Communication
Verified
Journal Article
Open access
CZCuicui Zhu
Cuicui Zhu
AWAishan Wumaier
Aishan Wumaier
DWDongping Wei
Dongping Wei
ZFZhixing Fan
Zhixing Fan
JYJianlei Yang
Jianlei Yang
LWLiejun Wang
Liejun Wang

5 October 2022 in IEEE Access
Verified
Journal Article
Open access
YSYunfei Shen
Yunfei Shen
QLQingqing Liu
Qingqing Liu
ZFZhixing Fan
Zhixing Fan
JLJiajun Liu
Jiajun Liu
AWAishan Wumaier
Aishan Wumaier
Mispronunciation Detection and Diagnosis (MDD) is an essential basic technology in Computer-Assisted Pronunciation Training (CAPT) and Computer-Assisted Language Learning (CALL). MDD research in Mandarin is faced with the problem of lack of relevant data, which is a typical low-resource scenario. In recent years, self-supervised pre-trained speech representation has developed rapidly and achieved significant performance improvement in low-resource speech recognition scenarios, making it necessary to be applied to MDD tasks. First, we build a Mandarin MDD dataset called PSC-Reading for the Putonghua Proficiency Test (PSC) passage reading section. Then we extended the end-to-end MDD system based on CTC/Attention hybrid architecture and Transformer architecture, using features extracted from self-supervised pre-training speech representation models such as Wav2Vec 2.0 and WavLM to replace conventional speech features like MFCC and Fbank, and conduct experiments on the PSC-Reading dataset. Experimental results show that, compared with the baseline model CNN-RNN-CTC, our WavLM-based model obtains 20.5% relative improvement on the F1 score metric.