9 June 2023 in Science Advances
Recent studies have shown that the feasibility of speech brain-computer interfaces (BCIs) as a clinically valid treatment in helping nontonal language patients with communication disorders restore their speech ability. However, tonal language speech BCI is challenging because additional precise control of laryngeal movements to produce lexical tones is required. Thus, the model should emphasize the features from the tonal-related cortex. Here, we designed a modularized multistream neural network that directly synthesizes tonal language speech from intracranial recordings. The network decoded lexical tones and base syllables independently via parallel streams of neural network modules inspired by neuroscience findings. The speech was synthesized by combining tonal syllable labels with nondiscriminant speech neural activity. Compared to commonly used baseline models, our proposed models achieved higher performance with modest training data and computational costs. These findings raise a potential strategy for approaching tonal language speech restoration.
30 October 2023 in Nature Communications
In tonal languages, which are spoken by nearly one-third of the world’s population, speakers precisely control the tension of vocal folds in the larynx to modulate pitch in order to distinguish words with completely different meanings. The specific pitch trajectories for a given tonal language are called lexical tones. Here, we used high-density direct cortical recordings to determine the neural basis of lexical tone production in native Mandarin-speaking participants. We found that instead of a tone category-selective coding, local populations in the bilateral laryngeal motor cortex (LMC) encode articulatory kinematic information to generate the pitch dynamics of lexical tones. Using a computational model of tone production, we discovered two distinct patterns of population activity in LMC commanding pitch rising and lowering. Finally, we showed that direct electrocortical stimulation of different local populations in LMC evoked pitch rising and lowering during tone production, respectively. Together, these results reveal the neural basis of vocal pitch control of lexical tones in tonal languages.
29 September 2026 in Nature Neuroscience
Mental imagery is a hallmark of human cognition, yet its neural mechanisms remain poorly understood. Speech imagery (the internal simulation of speech without overt articulation) has been proposed to partially share substrates with speech production, but its spatiotemporal dynamics remain controversial. Here, we leveraged high-resolution electrocorticography to investigate shared and modality-specific coding of articulatory kinematic trajectories during speech imagery and articulation. Linear modeling revealed robust articulatory kinematic trajectories encoding in frontoparietal cortex across modalities. Supramodal populations across middle premotor, subcentral and postcentral–supramarginal regions exhibited spatiotemporal stability during integrative planning. In contrast, modality-specific populations for speech imagery and articulation were somatotopically interleaved in primary sensorimotor cortex, revealing a distinct spatiotemporal organization. We further developed a generalized decoding framework that achieved high prediction accuracy for speech imagery (median 80.4%; chance 16.7%), comparable to speech articulation (78.3%). These findings uncover a somato-cognitive organization linking supramodal planning with modality-specific representations, informing imagery-based brain–computer interfaces.