Jiaming Zhou

Jiaming Zhou

Auto-generated profile, based on publicly available information

Jiaming Zhou

College of Computer Science, Nankai University, Tianjin, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 October 2025
Verified
Conference Paper
Open access
HWHui Wang
Hui Wang
SLShujie Liu
Shujie Liu
LMLingwei Meng
Lingwei Meng
JLJinyu Li
Jinyu Li
YYYifan Yang
Yifan Yang
YQYong Qin
Yong Qin

19 March 2025 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Closed access
HWHui Wang
Hui Wang
SZShiwan Zhao
Shiwan Zhao
XZXiguang Zheng
Xiguang Zheng
JZJiaming Zhou
Jiaming Zhou
XWXuechen Wang
Xuechen Wang
YQYong Qin
Yong Qin
Neural network-based automatic Mean Opinion Score (MOS) prediction plays a pivotal role in assessing the perceptual quality of synthetic speech. Historically, this field has struggled with limited data resources, a problem not fully resolved by existing methodologies. On the one hand, self-supervised learning (SSL) models could enhance the representational power of the feature extractor by extensive pre-training on large datasets, but fail to address the data scarcity problem affecting the downstream decoder. On the other hand, methods such as data augmentation and the incorporation of individual judges' assessments could improve data utilization, but hard to ensure a satisfactory balance between the scale and quality of the additional dataset. To address these challenges of decoder inefficiency and suboptimal data utilization, we introduce a retrieval-augmented MOS prediction method with prior knowledge integration. This method employs a new approach to using SSL features by retrieving similar instances in the feature space to obtain scores, thereby boosting decoder performance. We also optimize dataset utilization by exploiting quality-related information from the dataset as prior knowledge. Furthermore, our framework includes a dynamic retrieval scope adaptation and a fusion network, enhancing both robustness and accuracy while minimizing the need for manual parameter tuning. Experimental results demonstrate that our retrieval approach, enhanced by prior knowledge, significantly outperforms existing state-of-the-art methods in the synthetic speech MOS prediction task across various scenarios. Our code and checkpoints are available at https://github.com/NKU-HLT/RAMP_MOS.git.

1 January 2025
Verified
Conference Paper
Open access
PDF file
JZJiaming Zhou
Jiaming Zhou
SWShiyao Wang
Shiyao Wang
SZShiwan Zhao
Shiwan Zhao
JHJiabei He
Jiabei He
HSHaoqin Sun
Haoqin Sun
YQYong Qin
Yong Qin
Jiaming Zhou, Shiyao Wang, Shiwan Zhao, Jiabei He, Haoqin Sun, Hui Wang, Cheng Liu, Aobo Kong, Yujie Guo, Xi Yang, Yequan Wang, Yonghua Lin, Yong Qin. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

5 December 2025 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Closed access
HSHaoqin Sun
Haoqin Sun
XLXugang Lu
Xugang Lu
JTJingguang Tian
Jingguang Tian
JZJiaming Zhou
Jiaming Zhou
JHJiabei He
Jiabei He
YQYong Qin
Yong Qin
Multimodal emotion recognition (MMER) is crucial for effective human-computer interaction, yet its performance degrades sharply when modality data are incomplete due to privacy constraints or data loss. Existing approaches often attempt to reconstruct missing information directly from raw, heterogeneous modalities, requiring complex cross-modal mappings that lead to low-quality reconstruction and hence less performance improvement. To address these challenges, we propose a progressive learning framework (ProLF) for missing-modality reconstruction and robust multimodal fusion. ProLF’s core innovation lies in its design of an incremental three-stage strategy, i.e., Alignment, Reconstruction, and Augmentation-which collectively mitigates modality heterogeneity, data incompleteness, and weak fused representations. In the Alignment stage, ProLF applies unsupervised distribution-level contrastive learning to reduce cross-modal discrepancies and establish a unified latent space. In the Reconstruction stage, a normalizing flow model efficiently transforms the aligned latent distributions to recover missing modalities with high accuracy. In the Augmentation stage, a mixture of modality experts performs deep fusion of the restored multimodal information, while supervised point-based contrastive learning enhances emotion-specific features and suppresses irrelevant semantics, substantially improving discriminability. Extensive experiments on multiple benchmark datasets validate the effectiveness of ProLF. On IEMOCAP and MSP-IMPROV, ProLF achieves average performance gains of 1.5%–2%, and on CMU-MOSI, improvements of 1%–3%. These results confirm that ProLF provides robust and consistent benefits under both missing and complete modality conditions.

26 July 2024 in arXiv
Verified
Preprint
Open access
PDF file
SWShiyao Wang
Shiyao Wang
SZShiwan Zhao
Shiwan Zhao
JZJiaming Zhou
Jiaming Zhou
AKAobo Kong
Aobo Kong
YQYong Qin
Yong Qin
Dysarthric speech recognition (DSR) presents a formidable challenge due to inherent inter-speaker variability, leading to severe performance degradation when applying DSR models to new dysarthric speakers. Traditional speaker adaptation methodologies typically involve fine-tuning models for each speaker, but this strategy is cost-prohibitive and inconvenient for disabled users, requiring substantial data collection. To address this issue, we introduce a prototype-based approach that markedly improves DSR performance for unseen dysarthric speakers without additional fine-tuning. Our method employs a feature extractor trained with HuBERT to produce per-word prototypes that encapsulate the characteristics of previously unseen speakers. These prototypes serve as the basis for classification. Additionally, we incorporate supervised contrastive learning to refine feature extraction. By enhancing representation quality, we further improve DSR performance, enabling effective personalized DSR. We release our code at https://github.com/NKU-HLT/PB-DSR.

19 October 2025 in arXiv
Verified
Preprint
Open access
PDF file
SWShiyao Wang
Shiyao Wang
SZShiwan Zhao
Shiwan Zhao
JZJiaming Zhou
Jiaming Zhou
YQYong Qin
Yong Qin
Dysarthric speech recognition (DSR) research has witnessed remarkable progress in recent years, evolving from the basic understanding of individual words to the intricate comprehension of sentence-level expressions, all driven by the pressing communication needs of individuals with dysarthria. Nevertheless, the scarcity of available data remains a substantial hurdle, posing a significant challenge to the development of effective sentence-level DSR systems. In response to this issue, dysarthric data augmentation (DDA) has emerged as a highly promising approach. Generative models are frequently employed to generate training data for automatic speech recognition tasks. However, their effectiveness hinges on the ability of the synthesized data to accurately represent the target domain. The wide-ranging variability in pronunciation among dysarthric speakers makes it extremely difficult for models trained on data from existing speakers to produce useful augmented data, especially in zero-shot or one-shot learning settings. To address this limitation, we put forward a novel text-coverage strategy specifically designed for text-matching data synthesis. This innovative strategy allows for efficient zero/one-shot DDA, leading to substantial enhancements in the performance of DSR when dealing with unseen dysarthric speakers. Such improvements are of great significance in practical applications, including dysarthria rehabilitation programs and day-to-day common-sentence communication scenarios.