Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eye-tracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model.

Parkinson’s Disease (PD) is among the most frequent neurological disorders. Approaches that employ artificial intelligence and notably deep learning, have been extensively embraced with promising outcomes. This study dispenses an exhaustive review between 2016 and January 2023 on deep learning techniques used in the prognosis and evolution of symptoms and characteristics of the disease based on gait, upper limb movement, speech and facial expression-related information as well as the fusion of more than one of the aforementioned modalities. The search resulted in the selection of 87 original research publications, of which we have summarized the relevant information regarding the utilized learning and development process, demographic information, primary outcomes, and sensory equipment related information. Various deep learning algorithms and frameworks have attained state-of-the-art performance in many PD-related tasks by outperforming conventional machine learning approaches, according to the research reviewed. In the meanwhile, we identify significant drawbacks in the existing research, including a lack of data availability and interpretability of models. The fast advancements in deep learning and the rise in accessible data provide the opportunity to address these difficulties in the near future and for the broad application of this technology in clinical settings.

Gait analysis is crucial for the detection and management of various neurological and musculoskeletal disorders. The identification of gait events is valuable for enhancing gait analysis, developing accurate monitoring systems, and evaluating treatments for pathological gait. The aim of this work is to introduce the Smart-Insole Dataset to be used for the development and evaluation of computational methods focusing on gait analysis. Towards this objective, temporal and spatial characteristics of gait have been estimated as the first insight of pathology. The Smart-Insole dataset includes data derived from pressure sensor insoles, while 29 participants (healthy adults, elderly, Parkinson’s disease patients) performed two different sets of tests: The Walk Straight and Turn test, and a modified version of the Timed Up and Go test. A neurologist specialized in movement disorders evaluated the performance of the participants by rating four items of the MDS-Unified Parkinson’s Disease Rating Scale. The annotation of the dataset was performed by a team of experienced computer scientists, manually and using a gait event detection algorithm. The results evidence the discrimination between the different groups, and the verification of established assumptions regarding gait characteristics of the elderly and patients suffering from Parkinson’s disease.

The cognitive workload is an important component in performance psychology, ergonomics, and human factors. Publicly available datasets are scarce, making it difficult to establish new approaches and comparative studies. In this work, COLET-COgnitive workLoad estimation based on Eye-Tracking dataset is presented. Forty-seven (47) individuals' eye movements were monitored as they solved puzzles involving visual search activities of varying complexity and duration. The participants' cognitive workload level was evaluated with the subjective test of NASA-TLX and this score is used as an annotation of the activity. Extensive data analysis was performed in order to derive eye and gaze features from low-level eye recorded metrics, and a range of machine learning models were evaluated and tested regarding the estimation of the cognitive workload level. The activities induced four different levels of cognitive workload. Multi tasking and time pressure have induced a higher level of cognitive workload than the one induced by single tasking and absence of time pressure. Multi tasking had a significant effect on 17 eye features while time pressure had a significant effect on 7 eye features. Both binary and multi-class identification attempts were performed by testing a variety of well-known classifiers, resulting in encouraging results towards cognitive workload levels estimation, with up to 88% correct predictions between low and high cognitive workload. Machine learning analysis demonstrated potential in discriminating cognitive workload levels using only eye-tracking characteristics. The proposed dataset includes a much higher sample size and a wider spectrum of eye and gaze metrics than other similar datasets, allowing for the examination of their relations with various cognitive states.

Parkinson’s disease (PD) is characterized by a variety of motor and non-motor symptoms, some of them pertaining to gait and balance. The use of sensors for the monitoring of patients’ mobility and the extraction of gait parameters, has emerged as an objective method for assessing the efficacy of their treatment and the progression of the disease. To that end, two popular solutions are pressure insoles and body-worn IMU-based devices, which have been used for precise, continuous, remote, and passive gait assessment. In this work, insole and IMU-based solutions were evaluated for assessing gait impairment, and were subsequently compared, producing evidence to support the use of instrumentation in everyday clinical practice. The evaluation was conducted using two datasets, generated during a clinical study, in which patients with PD wore, simultaneously, a pair of instrumented insoles and a set of wearable IMU-based devices. The data from the study were used to extract and compare gait features, independently, from the two aforementioned systems. Subsequently, subsets comprised of the extracted features, were used by machine learning algorithms for gait impairment assessment. The results indicated that insole gait kinematic features were highly correlated with those extracted from IMU-based devices. Moreover, both had the capacity to train accurate machine learning models for the detection of PD gait impairment.

Parkinson’s disease (PD) is one of the most prevalent neurological diseases, described by complex clinical phenotypes. The manifestations of PD include both motor and non-motor symptoms. We constituted an experimental protocol for the assessment of PD motor signs of lower extremities. Using a pair of sensor insoles, data were recorded from PD patients, Elderly and Adult groups. Assessment of PD patients has been performed by neurologists specialized in movement disorders using the Movement Disorder Society—Unified Parkinson’s Disease Rating Scale (MDS-UPDRS)-Part III: Motor Examination, on both ON and OFF medication states. Using as a reference point the quantified metrics of MDS-UPDRS-Part III, severity levels were explored by classifying normal, mild, moderate, and severe levels of PD. Elaborating the recorded gait data, 18 temporal and spatial characteristics have been extracted. Subsequently, feature selection techniques were applied to reveal the dominant features to be used for four classification tasks. Specifically, for identifying relations between the spatial and temporal gait features on: PD and non-PD groups; PD, Elderly and Adults groups; PD and ON/OFF medication states; MDS-UPDRS: Part III and PD severity levels. AdaBoost, Extra Trees, and Random Forest classifiers, were trained and tested. Results showed a recognition accuracy of 88%, 73% and 81% for, the PD and non-PD groups, PD-related medication states, and PD severity levels relevant to MDS-UPDRS: Part III ratings, respectively.