Restoration of speech communication for locked-in patients by means of brain computer interfaces (BCIs) is currently an important area of active research. Among the neural signals obtained from intracranial recordings, single/multi-unit activity (SUA/MUA), local field potential (LFP), and electrocorticography (ECoG) are good candidates for an input signal for BCIs. However, the question of which signal or which combination of the three signal modalities is best suited for decoding speech production remains unverified. In order to record SUA, LFP, and ECoG simultaneously from a highly localized area of human ventral sensorimotor cortex (vSMC), we fabricated an electrode the size of which was 7 by 13 mm containing sparsely arranged microneedle and conventional macro contacts. We determined which signal modality is the most capable of decoding speech production, and tested if the combination of these signals could improve the decoding accuracy of spoken phonemes. Feature vectors were constructed from spike frequency obtained from SUAs and event-related spectral perturbation derived from ECoG and LFP signals, then input to the decoder. The results showed that the decoding accuracy for five spoken vowels was highest when features from multiple signals were combined and optimized for each subject, and reached 59% when averaged across all six subjects. This result suggests that multi-scale signals convey complementary information for speech articulation. The current study demonstrated that simultaneous recording of multi-scale neuronal activities could raise decoding accuracy even though the recording area is limited to a small portion of cortex, which is advantageous for future implementation of speech-assisting BCIs.

Decoding the inner representation of a word meaning from human cortical activity is a substantial challenge in the development of speech brain–machine interfaces (BMIs). The semantic aspect of speech is a novel target of speech decoding that may enable versatile communication platforms for individuals with impaired speech ability; however, there is a paucity of electrocorticography studies in this field. We decoded the semantic representation of a word from single-trial cortical activity during an imageability-based property identification task that required participants to discriminate between the abstract and concrete words. Using high gamma activity in the language-dominant hemisphere, a support vector machine classifier could discriminate the 2-word categories with significantly high accuracy (73.1 ± 7.5%). Activities in specific time components from two brain regions were identified as significant predictors of abstract and concrete dichotomy. Classification using these feature components revealed that comparable prediction accuracy could be obtained based on a spatiotemporally targeted decoding approach. Our study demonstrated that mental representations of abstract and concrete word processing could be decoded from cortical high gamma activities, and the coverage of implanted electrodes and time window of analysis could be successfully minimized. Our findings lay the foundation for the future development of semantic-based speech BMIs.