Current technological solutions for Brain-machine Interfaces (BMI) achieve reasonable accuracy, but most systems are large in size, power consuming and not auto-adaptive. This work addresses the question whether current neuromorphic technologies could resolve these problems? The paper proposes a novel neuromorphic framework of a BMI system for prosthetics control via decoding Electro Cortico-Graphic (ECoG) brain signals. It includes a three-dimensional spike timing neural network (3D-SNN) for brain signals features extraction and an on-line trainable recurrent reservoir structure (Echo state network (ESN)) for Motor Control Decoding (MCD). A software system, written in Python using NEST Simulator SNN library is described. It is able to adapt continuously in real time in supervised or unsupervised mode. The proposed approach was tested on several experimental data sets acquired from a tetraplegic person. First simulation results are encouraging, showing also the need for a further improvement via multiple hyper-parameters tuning. Its future implementation on a neuromorphic hardware platform that is smaller in size and significantly less power consuming is discussed too.

Current technological solutions for Brain-machine Interfaces (BMI) achieve reasonable accuracy, but most systems are large in size, power consuming and not auto-adaptive. This work addresses the question whether current neuromorphic technologies could resolve these problems? The paper proposes a novel neuromorphic framework of a BMI system for prosthetics control via decoding Electro Cortico-Graphic (ECoG) brain signals. It includes a three-dimensional spike timing neural network (3D-SNN) for brain signals features extraction and an on-line trainable recurrent reservoir structure (Echo state network (ESN)) for Motor Control Decoding (MCD). A software system, written in Python using NEST Simulator SNN library is described. It is able to adapt continuously in real time in supervised or unsupervised mode. The proposed approach was tested on several experimental data sets acquired from a tetraplegic person. First simulation results are encouraging, showing also the need for a further improvement via multiple hyper-parameters tuning. Its future implementation on a neuromorphic hardware platform that is smaller in size and significantly less power consuming is discussed too.

This work is demonstrating the use of a supercomputer platform to optimise hyper-parameters of a proposed by the team novel SNN-ESN computational model, that combines a brain template of spiking neurons in a spiking neural network (SNN) for feature extraction and an Echo State Network (ESN) for dynamic data series classification. A case study problem and data are used to illustrate the functionalities of the SNN-ESN. The overall SNN-ESN classifier has several hyper-parameters that are subject to refinement, such as: spiking threshold, duration of the refractory period and STDP learning rate for the SNN part; reservoir size, spectral radius of the connectivity matrix and leaking rate for the ESN part. In order to find the optimal hyper-parameter values exhaustive search over all possible combinations within reasonable intervals was performed using supercomputer Avitohol. The resulted optimal parameters led to improved classification accuracy. This work demonstrates the importance of model parameter optimisation using a supercomputer platform, which improves the usability of the proposed SNN-ESN for real-time applications on complex spatio-temporal data.

In our previous work we developed a neuromorphic decoder of intended movements of tetraplegic patients using ECoG recordings from brain motor cortex called Motor Control Decoder (MCD). Even though the training data was labeled based on the desired movement, there is no guarantee that the patient is satisfied by the action of the effectors. Hence the need for classification of brain signals as satisfactory/unsatisfactory is obvious. Based on previous work here we upgrade our neuromorphic MCD with a Neural Response Decoder (NRD) that is intended to predict whether ECoG data is satisfactory or not in order to improve MCD accuracy. The main aim is to design an actor-critic structure able to adapt via reinforcement learning the MCD (actor) based on NRD (critic) predictions. For this aim NRD was trained using not only ECoG signal but also the MCD prediction or prescribed intended movement of the patient. The achieved accuracy of trained NRD is satisfactory and it contributes to improved MCD performance. However, further work has to be done to fully utilize the NRD for MCD performance optimization in on-line manner. Possibility to include feedback from the patient would allow for further improvement of MCD-NRD accuracy.

In our previous work, we developed a neuromorphic decoder of intended movements of tetraplegic patients using ECoG recordings from the brain motor cortex, called Motor Control Decoder (MCD). Even though the training data are labeled based on the desired movement, there is no guarantee that the patient is satisfied by the action of the effectors. Hence, the need for the classification of brain signals as satisfactory/unsatisfactory is obvious. Based on previous work, we upgrade our neuromorphic MCD with a Neural Response Decoder (NRD) that is intended to predict whether ECoG data are satisfactory or not in order to improve MCD accuracy. The main aim is to design an actor–critic structure able to adapt via reinforcement learning the MCD (actor) based on NRD (critic) predictions. For this aim, NRD was trained using not only an ECoG signal but also the MCD prediction or prescribed intended movement of the patient. The achieved accuracy of the trained NRD is satisfactory and contributes to improved MCD performance. However, further work has to be carried out to fully utilize the NRD for MCD performance optimization in an on-line manner. Possibility to include feedback from the patient would allow for further improvement of MCD-NRD accuracy.