Using multimodal signals to solve the problem of emotion recognition is one of the emerging trends in affective computing. Several studies have utilized state of the art deep learning methods and combined physiological signals, such as the electrocardiogram (EEG), electroencephalogram (ECG), skin temperature, along with facial expressions, voice, posture to name a few, in order to classify emotions. Spiking neural networks (SNNs) represent the third generation of neural networks and employ biologically plausible models of neurons. SNNs have been shown to handle Spatio-temporal data, which is essentially the nature of the data encountered in emotion recognition problem, in an efficient manner. In this work, for the first time, we propose the application of SNNs in order to solve the emotion recognition problem with the multimodal dataset. Specifically, we use the NeuCube framework, which employs an evolving SNN architecture to classify emotional valence and evaluate the performance of our approach on the MAHNOB-HCI dataset. The multimodal data used in our work consists of facial expressions along with physiological signals such as ECG, skin temperature, skin conductance, respiration signal, mouth length, and pupil size. We perform classification under the Leave-One-Subject-Out (LOSO) cross-validation mode. Our results show that the proposed approach achieves an accuracy of 73.15% for classifying binary valence when applying feature-level fusion, which is comparable to other deep learning methods. We achieve this accuracy even without using EEG, which other deep learning methods have relied on to achieve this level of accuracy. In conclusion, we have demonstrated that the SNN can be successfully used for solving the emotion recognition problem with multimodal data and also provide directions for future research utilizing SNN for Affective computing. In addition to the good accuracy, the SNN recognition system is requires incrementally trainable on new data in an adaptive way. It only one pass training, which makes it suitable for practical and on-line applications. These features are not manifested in other methods for this problem.

The application of artificial intelligence (AI) and machine learning (ML) in biomedical research promises to unlock new information from the vast amounts of data being generated through the delivery of healthcare and the expanding high-throughput research applications. Such information can aid medical diagnoses and reveal various unique patterns of biochemical and immune features that can serve as early disease biomarkers. In this report, we demonstrate the feasibility of using an AI/ML approach in a relatively small dataset to discriminate among three categories of samples obtained from mice that either rejected or tolerated their pancreatic islet allografts following transplant in the anterior chamber of the eye, and from naïve controls. We created a locked software based on a support vector machine (SVM) technique for pattern recognition in electropherograms (EPGs) generated by micellar electrokinetic chromatography and laser induced fluorescence detection (MEKC-LIFD). Predictions were made based only on the aligned EPGs obtained in microliter-size aqueous humor samples representative of the immediate local microenvironment of the islet allografts. The analysis identified discriminative peaks in the EPGs of the three sample categories. Our classifier software was tested with targeted and untargeted peaks. Working with the patterns of untargeted peaks (i.e., based on the whole pattern of EPGs), it was able to achieve a 21 out of 22 positive classification score with a corresponding 95.45% prediction accuracy among the three sample categories, and 100% accuracy between the rejecting and tolerant recipients. These findings demonstrate the feasibility of AI/ML approaches to classify small numbers of samples and they warrant further studies to identify the analytes/biochemicals corresponding to discriminative features as potential biomarkers of islet allograft immune rejection and tolerance.

Objective. The classical ERP-based speller, or P300 Speller, is one of the most commonly used paradigms in the field of Brain Computer Interfaces (BCI). Several alterations to the visual stimuli presentation system have been developed to avoid unfavorable effects elicited by adjacent stimuli. However, there has been little, if any, regard to useful information contained in responses to adjacent stimuli about spatial location of target symbols. This paper aims to demonstrate that combining the classification of non-target adjacent stimuli with standard classification (target versus non-target) significantly improves classical ERP-based speller efficiency. Approach. Four SWLDA classifiers were trained and combined with the standard classifier: the lower row, upper row, right column and left column classifiers. This new feature extraction procedure and the classification method were carried out on three open databases: the UAM P300 database (Universidad Autonoma Metropolitana, Mexico), BCI competition II (dataset IIb) and BCI competition III (dataset II). Main results. The inclusion of the classification of non-target adjacent stimuli improves target classification in the classical row/column paradigm. A gain in mean single trial classification of 9.6% and an overall improvement of 25% in simulated spelling speed was achieved. Significance. We have provided further evidence that the ERPs produced by adjacent stimuli present discriminable features, which could provide additional information about the spatial location of intended symbols. This work promotes the searching of information on the peripheral stimulation responses to improve the performance of emerging visual ERP-based spellers.