4 January 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Learning fixed-dimensional speaker representation using deep neural networks is a key step in speaker verification. In this work, we propose an auxiliary memory storable network (MSN) to assist a backbone network for learning discriminative features, which are sequentially aggregated from lower to deeper layers of the backbone. The proposed MSN has a similar architecture to the ResNet and contains a set of cascaded feature aggregation (FA) blocks. Each FA block first aggregates the multi-level features from the previous block and the features from the corresponding backbone layer. The output features of each intermediate layer within the backbone are then refined by the multi-level features of the corresponding FA block through masking and biasing operations. Finally, the features from the last layers of both MSN and the backbone are concatenated to form more discriminative speaker representations. Experimental results on five public datasets show significant and consistent improvements over conventional approaches. The effectiveness of the proposed method is also validated using ablation studies, showing a robust generalization capacity in combination with different backbone networks.
16 December 2021 in IEEE Signal Processing Letters
Various mismatchedconditions result in performance degradation of the speaker verification (SV) systems. To address this issue, we extract robust speaker representations by devising a global-local information-based dynamic convolution neural network. In the proposed method, both global and local information of the input features are exploited to dynamically modify the convolution kernel values. This increases the model capability of capturing speaker characteristics by compensating both the inter- and intra-session variabilities. Extensive experiments on four publicly available SV datasets show significant and consistent improvements over the conventional approaches. The effectiveness of the proposed method is further investigated using ablation studies and visualizations.
13 September 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Speaker verification (SV) has suffered from session variability in complex acoustic scenarios, and learning session independent speaker representations remains a challenging problem. To tackle this, we propose a dynamic convolution framework for SV in this article, which dynamically adapts the model parameters to each input feature during inference, such that the model can flexibly extract robust speaker characteristics under different acoustic conditions. Specifically, we combine an adaptive context vector extraction (ACVE) module and a sub-kernel scaling (SKS) module in a cascaded manner. The ACVE uses a moving weighted sum and a sub-band self-attention in parallel to extract global-local information, which is then passed through the subsequent SKS module for efficient dynamic kernel generation. The proposed method can be easily implemented on the backbone network by replacing the conventional static counterparts. Experimental results on five public SV datasets show significant and consistent improvements over comparison approaches, and ablation studies and visual analysis further demonstrate the effectiveness of the proposed method.
12 March 2026 in IEEE Signal Processing Letters
Robust speaker verification under noisy conditions remains an open challenge. Conventional deep learning methods learn a robust unified speaker representation space against diverse background noise and achieve significant improvement. In contrast, this paper presents a noise-conditioned mixture-of-experts framework that decomposes the feature space into specialized noise-aware subspaces for speaker verification. Specifically, we propose a noise-conditioned expert routing mechanism, a universal model based expert specialization strategy, and an SNR-decaying curriculum learning protocol, collectively improving model robustness and generalization under diverse noise conditions. The proposed method can automatically route inputs to expert networks based on noise information derived from the inputs, where each expert targets distinct noise characteristics while preserving speaker identity information. Comprehensive experiments demonstrate consistent superiority over baselines.