Speech production is a hierarchical mechanism involving the synchronization of the brain and the oral articulators, where the intention of linguistic concepts is transformed into meaningful sounds. Individuals with locked-in syndrome (fully paralyzed but aware) lose their motor ability completely including articulation and even eyeball movement. The neural pathway may be the only option to resume a certain level of communication for these patients. Current brain-computer interfaces (BCIs) use patients' visual and attentional correlates to build communication, resulting in a slow communication rate (a few words per minute). Direct decoding of imagined speech from the neural signals (and then driving a speech synthesizer) has the potential for a higher communication rate. In this study, we investigated the decoding of five imagined and spoken phrases from single-trial, non-invasive magnetoencephalography (MEG) signals collected from eight adult subjects. Two machine learning algorithms were used. One was an artificial neural network (ANN) with statistical features as the baseline approach. The other was convolutional neural networks (CNNs) applied on the spatial, spectral and temporal features extracted from the MEG signals. Experimental results indicated the possibility to decode imagined and spoken phrases directly from neuromagnetic signals. CNNs were found to be highly effective with an average decoding accuracy of up to 93% for the imagined and 96% for the spoken phrases.

Purpose. To develop a predictive model of speech loss in persons with amyotrophic lateral sclerosis (ALS) based on measures of respiratory, phonatory, articulatory, and resonatory functions that were selected using a data-mining approach.Method. Physiologic speech subsystem (respiratory, phonatory, articulatory, and resonatory) functions were evaluated longitudinally in 66 individuals with ALS using multiple instrumentation approaches including acoustic, aerodynamic, nasometeric, and kinematic. The instrumental measures of the subsystem functions were subjected to a principal component analysis and linear mixed effects models to derive a set of comprehensive predictors of bulbar dysfunction. These subsystem predictors were subjected to a Kaplan-Meier analysis to estimate the time until speech loss.Results. For a majority of participants, speech subsystem decline was detectible prior to declines in speech intelligibility and speaking rate. Among all subsystems, the articulatory and phonatory predictors were most responsive to early bulbar deterioration; and the resonatory and respiratory predictors were as responsive to bulbar decline as was speaking rate.Conclusions. The articulatory and phonatory predictors are sensitive indicators of early bulbar decline due to ALS, which has implications for predicting disease onset and progression and clinical management of ALS.

Silent speech recognition (SSR) converts nonaudio information such as articulatory movements into text. SSR has the potential to enable persons with laryngectomy to communicate through natural spoken expression. Current SSR systems have largely relied on speaker-dependent recognition models. The high degree of variability in articulatory patterns across different speakers has been a barrier for developing effective speaker-independent SSR approaches. Speaker-independent SSR approaches, however, are critical for reducing the amount of training data required from each speaker. In this paper, we investigate speaker-independent SSR from the movements of flesh points on tongue and lips with articulatory normalization methods that reduce the interspeaker variation. To minimize the across-speaker physiological differences of the articulators, we propose Procrustes matching-based articulatory normalization by removing locational, rotational, and scaling differences. To further normalize the articulatory data, we apply feature-space maximum likelihood linear regression and i-vector. In this paper, we adopt a bidirectional long short-term memory recurrent neural network (BLSTM) as an articulatory model to effectively model the articulatory movements with long-range articulatory history. A silent speech dataset with flesh-point articulatory movements was collected using an electromagnetic articulograph from 12 healthy and two laryngectomized English speakers. Experimental results showed the effectiveness of our speaker-independent SSR approaches on healthy as well as laryngectomy speakers. In addition, BLSTM outperformed the standard deep neural network. The best performance was obtained by the BLSTM with all the three normalization approaches combined.

Objective: This study aimed to determine the diagnostic utility of clinician speech ratings and patient self-report for detecting early bulbar changes associated with amyotrophic lateral sclerosis (ALS), compared to instrumentation-based speech measures. Methods: Thirty-six individuals with ALS and 17 healthy control participants were included. Patients’ awareness of early bulbar motor involvement was assessed using self-reported scores on the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R). Clinicians’ detection of early bulbar motor involvement was assessed through perceptual speech ratings by two experienced speech-language pathologists. Participants with ALS were grouped as ‘bulbar pre-symptomatic’ or ‘bulbar symptomatic’ based on self-report and clinician ratings, and compared to healthy controls on six instrumentation-based speech measures. ROC analysis was used to compare the sensitivity and specificity of perceptual and instrumentation-based measures for detecting bulbar changes in pre-symptomatic individuals. Results: Early bulbar changes that were documented using instrumentation-based measures were undetected by both patients and clinicians. ROC analyses indicated that instrumentation-based measures outperformed clinicians’ scaled severity ratings, and that percent pause time was the best measure for differentiating healthy controls from bulbar pre-symptomatic individuals with ALS. Conclusions: Findings suggested that instrumentation-based measures of speech may be necessary for early detection of bulbar changes due to ALS.

This paper addresses the problem of recognizing the speech uttered by patients with dysarthria, which is a motor speech disorder impeding the physical production of speech. Patients with dysarthria have articulatory limitation, and therefore, they often have trouble in pronouncing certain sounds, resulting in undesirable phonetic variation. Modern automatic speech recognition systems designed for regular speakers are ineffective for dysarthric sufferers due to the phonetic variation. To capture the phonetic variation, Kullback-Leibler divergence-based hidden Markov model (KL-HMM) is adopted, where the emission probability of state is parameterized by a categorical distribution using phoneme posterior probabilities obtained from a deep neural network-based acoustic model. To further reflect speaker-specific phonetic variation patterns, a speaker adaptation method based on a combination of L2 regularization and confusion-reducing regularization, which can enhance discriminability between categorical distributions of the KL-HMM states while preserving speaker-specific information is proposed. Evaluation of the proposed speaker adaptation method on a database of several hundred words for 30 speakers consisting of 12 mildly dysarthric, 8 moderately dysarthric, and 10 non-dysarthric control speakers showed that the proposed approach significantly outperformed the conventional deep neural network-based speaker adapted system on dysarthric as well as non-dysarthric speech.