Siyuan Wang

Siyuan Wang

Auto-generated profile, based on publicly available information

Siyuan Wang

Department of Electronic Engineering, Tsinghua University, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 October 2025
Verified
Conference Paper
Open access
ZYZhihang Yuan
Zhihang Yuan
SWSiyuan Wang
Siyuan Wang
YSYuzhang Shang
Yuzhang Shang
HZHanling Zhang
Hanling Zhang
TFTongcheng Fang
Tongcheng Fang
YWYu Wang
Yu Wang

14 July 2023
Verified
Conference Paper
Open access
HWHaoyu Wang
Haoyu Wang
SWSiyuan Wang
Siyuan Wang
YGYaguang Gong
Yaguang Gong
WZWei-Qiang Zhang
Wei-Qiang Zhang
Self-supervised pre-trained audio representation models such as Wav2vec or HuBERT have brought notable improvements to many downstream audio-related tasks, but the huge number of parameters of these pre-trained models sets a barrier to their application on memory-constrained edge devices. Recursive Transformers, represented by Albert, have proven that parameter sharing through transformer layers can obviously reduce the size of pre-trained models while maintaining most of the performance. In this paper, we propose DistilALHuBERT, a lightweight recursive transformer audio representation model distilled from Hubert. Evaluation results on the S3PRL benchmark show that DistilALHuBERT can significantly outperform the DistilHuBERT model with the same number of parameters. Our code and models are available at https://github.com/backspacetg/distilAlhubert.

14 July 2023
Verified
Conference Paper
Open access
SWSiyuan Wang
Siyuan Wang
HWHaoyu Wang
Haoyu Wang
JLJian Li
Jian Li
WZWei-Qiang Zhang
Wei-Qiang Zhang
Large-scale pre-trained models bring significant gains to many speech-related tasks. However, it is still challenging to use these large models when computing power of terminal equipment is limited. Pruning is an effective method to reduce memory footprint and cost calculation. The imperfect evaluation criteria of existing pruning methods and the complex fine tuning process result in a relatively high loss of accuracy. To solve these problems, we propose a structured pruning method, which introduced the upper confidence bound of importance scores to assess the potential of each component of the model more accurately. In addition, we also introduce a set of learnable pruning threshold parameters that can be learned via stochastic gradient descent, thereby reducing the hyper-parameter tuning. We apply our method to HuBERT models on automatic speech recognition (ASR) task. Our result shows that for all pruning granularity and pruning ratios, our method yields higher accuracy and speedup ratios in the inference process.When sparsity was 60%, our method performed only 0.63% down.