27 June 2025 in IEEE Signal Processing Letters
We explore unsupervised speech enhancement using diffusion models as expressive generative priors for clean speech. Existing approaches guide the reverse diffusion process using noisy speech through an approximate, noise-perturbed likelihood score, combined with the unconditional score via a trade-off hyperparameter. In this work, we propose two alternative algorithms that directly model the conditional reverse transition distribution of diffusion states. The first method integrates the diffusion prior with the observation model in a principled way, removing the need for hyperparameter tuning. The second defines a diffusion process over the noisy speech itself, yielding a fully tractable and exact likelihood score. Experiments on the WSJ0-QUT and VoiceBank-DEMAND datasets demonstrate improved enhancement metrics and greater robustness to domain shifts compared to both supervised and unsupervised baselines.
12 June 2026 in IEEE Transactions on Audio, Speech and Language Processing
This paper addresses unsupervised diffusion-based single-channel speech enhancement (SE). Prior work in this direction combines a score-based diffusion model trained on clean speech with a Gaussian noise model whose covariance is structured by non-negative matrix factorization (NMF). This combination is used within an iterative expectation-maximization (EM) scheme, in which a diffusion-based posterior-sampling E-step estimates the clean speech. We first revisit this framework and propose to explicitly model both speech and acoustic noise as latent variables, jointly sampling them in the E-step instead of sampling speech alone as in previous approaches. We then introduce a new semi-supervised SE framework that replaces the NMF noise prior with a diffusion-based noise model, learned jointly with the speech prior in a single conditional score model. Within this framework, we derive two variants: one that implicitly accounts for noise and one that explicitly treats noise as a latent variable. Experiments on WSJ0–QUT and VoiceBank–DEMAND show that explicit noise modeling systematically improves SE performance for both NMF-based and diffusion-based noise priors. Under matched conditions, the diffusion-based noise model attains the best overall quality and intelligibility among unsupervised methods, while under mismatched conditions the proposed NMF-based explicit-noise framework is more robust and suffers less degradation than several supervised baselines. Code, demo, and supplementary materials are publicly available.
6 November 2026 in bioRxiv
Cochlear and auditory brainstem implants are the primary treatments for profound deafness 1 . However, the inherently low resolution of electrical stimulation used by these devices limits information transfer to a small number of independent stimulation channels 2 and results in perceptual difficulties. An alternative approach is to target the auditory cortex, a brain region large enough to accommodate hundreds of stimulation channels and whose stimulation evokes sound percepts 3,4 . Although modern brain interfaces offer large channel counts and spatial resolution, developing encoding algorithms capable of accurately transmitting interpretable sound information remains challenging. Here we introduce Braincodec, a deep network that optimizes both sound information transmission and the alignment of stimulation patterns with the spatial, temporal, and high-level properties of the auditory cortex neural code. Braincodec operates online and requires only a few hundred channels to transmit sufficient information for the reconstruction of natural sounds with high fidelity, a performance inaccessible to cochlear implant encoders. Moreover, mice equipped with a Braincodec-driven auditory cortical prosthesis matched the accuracy of non-implanted mice in discriminating real sounds. Braincodec therefore enables the development of high-throughput cortical neuroprosthetics with the potential to overcome the limitations of current auditory implants and provides a generic framework applicable to other sensory modalities.