Haoqin Sun

Haoqin Sun

Auto-generated profile, based on publicly available information

Haoqin Sun

College of Computer Science, Nankai University, Tianjin, China

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 February 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
YLYang Liu
Yang Liu
HSHaoqin Sun
Haoqin Sun
WGWenbo Guan
Wenbo Guan
YXYuQi Xia
YuQi Xia
YLYongwei Li
Yongwei Li
ZZZhen Zhao
Zhen Zhao
Currently, speech emotion recognition models still could not show satisfactory performance due to the complexity of emotions. In most of the previous studies, there is a common problem that some of the particular emotions are severely misclassified. In this paper, we propose a novel framework integrating cascaded attention network and adversarial joint loss strategy for speech emotion recognition, aiming at discriminating the confusions by emphasizing more on the emotions which are difficult to be correctly classified. First, we extract log-Mels, deltas and delta-deltas of log-Mels as 3D features to effectively reduce the interference of external factors. Next, we introduce a cascaded attention network to extract effective emotional features, where spatiotemporal attention selectively locates the targeted emotional regions from the input features. In these targeted regions, the self attention with head fusion captures the long-distance dependence of temporal features. Finally, an adversarial joint loss strategy is proposed to distinguish the emotional embeddings with high similarity by the generated hard triplets in an adversarial fashion. To evaluate our proposed method, experiments are performed with the IEMOCAP, CASIA, and EMODB corpora. The experimental results demonstrate that our proposed method significantly outperforms the state-of-the-art approaches on all datasets.

27 October 2025
Verified
Conference Paper
Open access
HWHui Wang
Hui Wang
SLShujie Liu
Shujie Liu
LMLingwei Meng
Lingwei Meng
JLJinyu Li
Jinyu Li
YYYifan Yang
Yifan Yang
YQYong Qin
Yong Qin

1 June 2023 in Machine Intelligence Research
Verified
Journal Article
Closed access
PDF file
YLYang Liu
Yang Liu
HSHaoqin Sun
Haoqin Sun
WGWenbo Guan
Wenbo Guan
YXYuQi Xia
YuQi Xia
ZZZhen Zhao
Zhen Zhao

5 December 2025 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Closed access
HSHaoqin Sun
Haoqin Sun
XLXugang Lu
Xugang Lu
JTJingguang Tian
Jingguang Tian
JZJiaming Zhou
Jiaming Zhou
JHJiabei He
Jiabei He
YQYong Qin
Yong Qin
Multimodal emotion recognition (MMER) is crucial for effective human-computer interaction, yet its performance degrades sharply when modality data are incomplete due to privacy constraints or data loss. Existing approaches often attempt to reconstruct missing information directly from raw, heterogeneous modalities, requiring complex cross-modal mappings that lead to low-quality reconstruction and hence less performance improvement. To address these challenges, we propose a progressive learning framework (ProLF) for missing-modality reconstruction and robust multimodal fusion. ProLF’s core innovation lies in its design of an incremental three-stage strategy, i.e., Alignment, Reconstruction, and Augmentation-which collectively mitigates modality heterogeneity, data incompleteness, and weak fused representations. In the Alignment stage, ProLF applies unsupervised distribution-level contrastive learning to reduce cross-modal discrepancies and establish a unified latent space. In the Reconstruction stage, a normalizing flow model efficiently transforms the aligned latent distributions to recover missing modalities with high accuracy. In the Augmentation stage, a mixture of modality experts performs deep fusion of the restored multimodal information, while supervised point-based contrastive learning enhances emotion-specific features and suppresses irrelevant semantics, substantially improving discriminability. Extensive experiments on multiple benchmark datasets validate the effectiveness of ProLF. On IEMOCAP and MSP-IMPROV, ProLF achieves average performance gains of 1.5%–2%, and on CMU-MOSI, improvements of 1%–3%. These results confirm that ProLF provides robust and consistent benefits under both missing and complete modality conditions.

9 March 2026 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Closed access
JZJiaming Zhou
Jiaming Zhou
SZShiwan Zhao
Shiwan Zhao
HWHui Wang
Hui Wang
HSHaoqin Sun
Haoqin Sun
WZWenjia Zeng
Wenjia Zeng
YQYong Qin
Yong Qin
Retrieval-augmented methods complement parametric models with non-parametric memory and have shown strong adaptability in natural language processing. Extending this paradigm to automatic speech recognition (ASR) is appealing but challenging, particularly at the frame level where precise alignments are difficult to obtain and large-scale audio-text datastores impose heavy storage and retrieval costs. We present $k$NN-CTC $+$, a retrieval-augmented decoding framework for CTC-based ASR that addresses these challenges. To construct fine-grained datastores without external supervision, $k$ NN-CTC $+$ leverages CTC encoder embeddings as keys and frame-level pseudo labels as values. To reduce redundancy, we propose a blank-aware pruning strategy that removes frames during both datastore construction and query-time retrieval, improving efficiency while retaining informative evidence. Beyond alignment and scale issues, domain mismatch remains a major obstacle. $k$ NN-CTC $+$ introduces a lightweight unsupervised domain adaptation (UDA) pipeline: confidence-based filtering discards unreliable pseudo labels when building target-specific datastores, and a dynamic interpolation strategy adaptively balances model predictions and $k$ NN retrievals based on neighbor confidence. These designs enable robust test-time adaptation without labeled data or parameter updates. Experiments on mandarin, Chinese dialect, child and elderly speech datasets demonstrate consistent accuracy gains and improved robustness under domain shifts. $k$NN-CTC$+$ provides a scalable, effective solution for enhancing CTC-based ASR. Code is available at https://github.com/NKU-HLT/KNN-CTC.