Affective computing is a field of study that integrates human affects and emotions with artificial intelligence into systems or devices. A system or device with affective computing is beneficial for the mental health and wellbeing of individuals that are stressed, anguished, or depressed. Emotion recognition systems are an important technology that enables affective computing. Currently, there are a lot of ways to build an emotion recognition system using various techniques and algorithms. This review paper focuses on emotion recognition research that adopted electrocardiograms (ECGs) as a unimodal approach as well as part of a multimodal approach for emotion recognition systems. Critical observations of data collection, pre-processing, feature extraction, feature selection and dimensionality reduction, classification, and validation are conducted. This paper also highlights the architectures with accuracy of above 90%. The available ECG-inclusive affective databases are also reviewed, and a popularity analysis is presented. Additionally, the benefit of emotion recognition systems towards healthcare systems is also reviewed here. Based on the literature reviewed, a thorough discussion on the subject matter and future works is suggested and concluded. The findings presented here are beneficial for prospective researchers to look into the summary of previous works conducted in the field of ECG-based emotion recognition systems, and for identifying gaps in the area, as well as in developing and designing future applications of emotion recognition systems, especially in improving healthcare.

Affective computing focuses on instilling emotion awareness in machines. This area has attracted many researchers globally. However, the lack of an affective database based on physiological signals from the Asian continent has been reported. This is an important issue for ensuring inclusiveness and avoiding bias in this field. This paper introduces an emotion recognition database, the Asian Affective and Emotional State (A2ES) dataset, for affective computing research. The database comprises electrocardiogram (ECG) and photoplethysmography (PPG) recordings from 47 Asian participants of various ethnicities. The subjects were exposed to 25 carefully selected audio–visual stimuli to elicit specific targeted emotions. An analysis of the participants’ self-assessment and a list of the 25 stimuli utilised are also presented in this work. Emotion recognition systems are built using ECG and PPG data; five machine learning algorithms: support vector machine (SVM), k-nearest neighbour (KNN), naive Bayes (NB), decision tree (DT), and random forest (RF); and deep learning techniques. The performance of the systems built are presented and compared. The SVM was found to be the best learning algorithm for the ECG data, while RF was the best for the PPG data. The proposed database is available to other researchers.

This project involves the assessment of ECG feature extraction toolboxes for emotion recognition system. The objective of this work is to compare the performance between TEAP and AuBT by measuring the accuracy of the classification using the features extracted by each toolbox. Two publicly available datasets: DREAMER and AuBT dataset are used in this work. Only ECG signals from both datasets are extracted using TEAP and AuBT toolbox. Using support vector machine, the result for DREAMER dataset shows that the features extracted using TEAP provide better accuracy in classifying arousal while for valence, AuBT is better with 65.40% and 65.80% respectively. AuBT dataset result shows that AuBT toolbox performed marginally better than TEAP. Thus, for large dataset like DREAMER, the 13 features extracted using TEAP can be considered. However, for a smaller data size sample like the AuBT dataset, 81 features extracted by AuBT toolbox is found to benefit the classification process. Additionally, the result of arousal and valence of DREAMER also indicates that the type of emotion data may influence the suitability of the extracted features.