Yuchen Hu

Yuchen Hu

Auto-generated profile, based on publicly available information

Yuchen Hu

Nanyang Technological University (NTU)

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 December 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
YHYuchen Hu
Yuchen Hu
CCChen Chen
Chen Chen
QZQiushi Zhu
Qiushi Zhu
ECEng Siong Chng
Eng Siong Chng
Automatic speech recognition (ASR) has gained remarkable successes thanks to recent advances of deep learning, but it usually degrades significantly under real-world noisy conditions. Recent works introduce speech enhancement (SE) as front-end to improve speech quality, which is proved effective but may not be optimal for downstream ASR due to speech distortion problem. Based on that, latest works combine SE and currently popular self-supervised learning (SSL) to alleviate distortion and improve noise robustness. Despite the effectiveness, the speech distortion caused by conventional SE still cannot be cleared out. In this paper, we propose a self-supervised framework named Wav2code to implement a feature-level SE with reduced distortions for noise-robust ASR. First, in pre-training stage the clean speech representations from SSL model are sent to lookup a discrete codebook via nearest-neighbor feature matching, the resulted code sequence are then exploited to reconstruct the original clean representations, in order to store them in codebook as prior. Second, during finetuning we propose a Transformer-based code predictor to accurately predict clean codes by modeling global and local dependency of input noisy representations, which enables discovery and restoration of high-quality clean representations with reduced distortions. Furthermore, we propose an interactive feature fusion network to combine original noisy and the restored clean representations to consider both fidelity and quality, resulting in more informative features for downstream ASR. Finally, experiments on both synthetic and real noisy datasets demonstrate that Wav2code can solve the speech distortion and improve ASR performance under various noisy conditions, resulting in stronger robustness.

1 January 2026 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Closed access
XZXiaoying Zhao
Xiaoying Zhao
QZQiushi Zhu
Qiushi Zhu
YHYuchen Hu
Yuchen Hu
LDLirong Dai
Lirong Dai
Significant progress has been made in the field of speech enhancement in recent years, and most speech enhancement models denoise in either the time-frequency or time-domain; these methods rely on large amounts of paired clean-noisy speech data and may not generalize well on unseen noise types. Meanwhile, noise-augmented self-supervised representations can see more unlabeled data, showing strong denoising and generalization capabilities. How to more effectively utilize self-supervised representations for speech enhancement remains an open and challenging problem. In this paper, we first propose SERV, a speech enhancement framework based on noise-augmented self-supervised representations and vocoders, where the self-supervised model learns the mapping from speech waveforms to representations. By combining contrastive and consistency losses, the model learns noise-robust speech representations from large-scale noisy speech data. Second, in the fine-tuning stage, we build a vocoder that maps the learned representations back to speech and fine-tune the entire model using paired clean and noisy speech data. Finally, we conduct extensive experiments to evaluate the effects of different layers of representations and various types of self-supervised models. The experimental results demonstrate that our proposed method achieves strong performance and generalization.