Serialized Output Training (SOT) has emerged as the mainstream approach for addressing the multi-talker overlapped speech recognition challenge due to its simplicity. However, SOT encounters cross-domain performance degradation which hinders its application. Meanwhile, traditional domain adaption methods may harm the accuracy of speaker change point prediction evaluated by UD-CER, which is an important metric in SOT. To solve these issues, we propose Pseudo-Labeling based SOT (PL-SOT) for domain adaptation by treating speaker change token ( $< $ sc $>$ ) specially during training to increase the accuracy of speaker change point prediction. Firstly, we improve CTC loss by proposing Weakening and Enhancing CTC (WE-CTC) loss to weaken the learning of error-prone labels surrounding $< $ sc $>$ while enhance the emission probability of $< $ sc $>$ through modifying posteriors of the pseudo-labels. Secondly, we introduce Weighted Confidence Filter (WCF) that assigns higher scores of $< $ sc $>$ to exclude low-quality pseudo-labels without hurting the $< $ sc $>$ prediction. Experimental results show that PL-SOT achieves 17.7%/12.8% average relative reduction of CER/UD-CER, with AliMeeting as source domain and AISHELL-4 along with MagicData-RAMC as target domain.

Large Language Model (LLM) has been gradually adopted in Automatic Speech Recognition (ASR) task due to its strong long-context modeling capability, achieving remarkable performance. However, similar to other end-to-end ASR models, LLM-Based ASR (LLM-ASR) models encounter performance degradation when applied across domains. To address this issue, we propose LLM-Based Pseudo-Labeling (LPL), which leverages unlabeled audio data in the target domain to train a domain-adapted LLM-ASR model. Specifically, we propose Hybrid Pseudo-Label Decoding (HPLD) method to refine the quality of pseudo-labels in a quick non-autoregressive manner, achieving a balance between training performance and computational efficiency. Additionally, we introduce the CTC Assisted Prompt (CAP) method, which offers additional semantic hints to enhance the accuracy of transcriptions generated by the LLM. During training, we incorporate CTC loss at the post-projector position to align speech embedding prompt more effectively and feed masked CTC hypothesis embedding into the LLM prompt for further improvement. Experimental results demonstrate that LPL achieves state-of-the-art performance compared to other mainstream pseudo-labeling methods, further enhancing the cross-domain performance of LLM-ASR models.

Large language models (LLMs) have been increasingly applied in Automatic Speech Recognition (ASR), achieving significant advancements. However, the performance of LLM-based ASR (LLM-ASR) models remains unsatisfactory when applied across domains due to domain shifts between acoustic and linguistic conditions. To address this challenge, we propose a decoupled two-stage domain adaptation framework that separates the adaptation process into text-only and audio-only stages. In the first stage, we leverage abundant text data from the target domain to refine the LLM component, thereby improving its contextual and linguistic alignment with the target domain. In the second stage, we employ a pseudo-labeling method with unlabeled audio data in the target domain and introduce two key enhancements: (1) incorporating decoupled auxiliary Connectionist Temporal Classification (CTC) loss to improve the robustness of the speech encoder under different acoustic conditions; (2) adopting a synchronous LLM tuning strategy, allowing the LLM to continuously learn linguistic alignment from pseudo-labeled transcriptions enriched with domain textual knowledge. The experimental results demonstrate that our proposed methods significantly improve the performance of LLM-ASR in the target domain, achieving a relative word error rate reduction of 19.2%.