Mood disorders affect more than 300 million people worldwide and can cause devastating consequences. Elderly people and patients with neurological conditions are particularly susceptible to depression. Gait and body movements can be affected by mood disorders, and thus they can be used as a surrogate sign, as well as an objective index for pervasive monitoring of emotion and mood disorders in daily life. Here we review evidence that demonstrates the relationship between gait, emotions and mood disorders, highlighting the potential of a multimodal approach that couples gait data with physiological signals and home-based monitoring for early detection and management of mood disorders. This could enhance self-awareness, enable the development of objective biomarkers that identify high risk subjects and promote subject-specific treatment.

Finite/fixed-time control yields a promising tool to optimize a system's settling time, but lacks the ability to separately define the settling time and the convergence domain (known as practically prescribed-time stability , PPTS). We provide a sufficient condition for PPTS based on a new piecewise exponential function, which decouples the settling time and convergence domain into separately user-defined parameters. We propose an adaptive event-triggered prescribed-time control scheme for nonlinear systems with asymmetric output constraints, using an exponential-type barrier Lyapunov function. We show that this PPTS control scheme can guarantee tracking error convergence performance, while restricting the output state according to the prescribed asymmetric constraints. Compared with traditional finite/fixed-time control, the proposed methodology yields separately user-defined settling time and convergence domain without the prior information on disturbance. Moreover, asymmetric state constraints can be handled in the control structure through bias state transformation, which offers an intuitive analysis technique for general constraint issues. Simulation and experiment results on a heterogeneous teleoperation system demonstrate the merits of the proposed control scheme.

Assistive robots play an important role in improving the quality of life of patients at home. Among all the monitoring tasks, gait disorders are prevalent in elderly and people with neurological conditions and this increases the risk of fall. Therefore, the development of mobile systems for gait monitoring at home in normal living conditions is important. Here, we present a mobile system that is able to track humans and analyze their gait in canonical coordinates based on a single RGB-D camera. First, view-invariant three-dimensional (3-D) lower limb pose estimation is achieved by fusing information from depth images along with 2-D joints derived in RGB images. Next, both the 6-D camera pose and the 3-D lower limb skeleton are real-time tracked in a canonical coordinate system based on simultaneously localization and mapping (SLAM). A mask-based strategy is exploited to improve the re-localization of the SLAM in dynamic environments. Abnormal gait is detected by using the support vector machine and the bidirectional long-short term memory network with respect to a set of extracted gait features. To evaluate the robustness of the system, we collected multi-cameras, ground truth data from 16 healthy volunteers performing 6 gait patterns that mimic common gait abnormalities. The experiment results demonstrate that our proposed system can achieve good lower limb pose estimation and superior recognition accuracy compared to previous abnormal gait detection methods.

Monitoring the mental workload of operators is of paramount importance in space telerobotic training and other teleoperation tasks. Instead of the estimation of task-specific workload, this article aims at investigating the impact of two significant confounding factors (time-pressure and latency) on space teleoperation and explored the use of eye-tracking technology for factor-induced mental workload estimation and performance evaluation. Ten subjects teleoperated a Canadarm2 robot to complete a complex on-orbit assembly task in our photo-realistic training simulator while wearing a head-mounted eye-tracker. To understand how time-pressure and latency influence eye-tracking features works, we first performed the statistical analysis on various features with respect to a single factor and across multiple groups. Next, eye-tracking features extracted from segment data and trial data is used to identify the mental workload induced by confounding factors, which can be used for developing personalized training programs and guaranteeing safe teleoperation. Furthermore, to improve the recognition performance using segment data, we propose the activity ratio and time ratio to characterize the informative segments. Finally, the relationship between simulator-defined performance measures and eye-tracking features is examined. Results show that fixation duration, saccade frequency and duration, pupil diameter, and index of pupillary activity are significant features that can be used in both factor-induced mental workload estimation and task performance evaluation.

For abnormal gait recognition, pattern-specific features indicating abnormalities are interleaved with the subject-specific differences representing biometric traits. Deep representations are, therefore, prone to overfitting, and the models derived cannot generalize well to new subjects. Furthermore, there is limited availability of abnormal gait data obtained from precise Motion Capture (Mocap) systems because of regulatory issues and slow adaptation of new technologies in health care. On the other hand, data captured from markerless vision sensors or wearable sensors can be obtained in home environments, but noises from such devices may prevent the effective extraction of relevant features. To address these challenges, we propose a cascade of deep architectures that can encode cross-modal and cross-subject transfer for abnormal gait recognition. Cross-modal transfer maps noisy data obtained from RGBD and wearable sensors to accurate 4-D representations of the lower limb and joints obtained from the Mocap system. Subsequently, cross-subject transfer allows disentangling subject-specific from abnormal pattern-specific gait features based on a multiencoder autoencoder architecture. To validate the proposed methodology, we obtained multimodal gait data based on a multicamera motion capture system along with synchronized recordings of electromyography (EMG) data and 4-D skeleton data extracted from a single RGBD camera. Classification accuracy was improved significantly in both Mocap and noisy modalities.

Egocentric vision has gained increasing popularity recently, opening new avenues for human-centric applications. However, the use of the egocentric fisheye cameras allows wide angle coverage but image distortion is introduced along with strong human body self-occlusion imposing significant challenges in data processing and model reconstruction. Unlike previous work only leveraging synthetic data for model training, this paper presents a new real-world EgoCentric Human Pose (ECHP) dataset. To tackle the difficulty of collecting 3D ground truth using motion capture systems, we simultaneously collect images from a head-mounted egocentric fisheye camera as well as from two third-person-view cameras, circumventing the environmental restrictions. By using self-supervised learning under multi-view constraints, we propose a simple yet effective framework, namely EgoFish3D, for egocentric 3D pose estimation from a single image in different real-world scenarios. The proposed EgoFish3D incorporates three main modules. 1) The third-person-view module takes two exocentric images as input and estimates the 3D pose represented in the third-person camera frame; 2) the egocentric module predicts the 3D pose in the egocentric camera frame; and 3) the interactive module estimates the rotation matrix between the third-person and the egocentric views. Experimental results on our ECHP dataset and existing benchmark datasets demonstrate the effectiveness of the proposed EgoFish3D, which can achieve superior performance to existing methods.