Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work.

We address the problem of facial expression analysis. The proposed approach predicts both basic emotion and valence/arousal values as a continuous measure for the emotional state. Experimental results including cross-database evaluation on the AffectNet, Aff-Wild, and AFEW dataset shows that our approach predicts emotion categories and valence/arousal values with high accuracies and that the simultaneous learning of discrete categories and continuous values improves the prediction of both. In addition, we use our approach to measure the emotional states of users in an Human-Robot-Collaboration scenario (HRC), show how these emotional states are affected by multiple difficulties that arise for the test subjects, and examine how different feedback mechanisms counteract negative emotions users experience while interacting with a robot system.

The automated analysis of facial expression is a long investigated subject in the computer vision community and has been boosted by applications in the field of human computer interaction (HCI). Besides mapping of facial expressions to basic emotion categories, what is often of limited use for HCI due to sparse occurrence of real emotions, other approaches have been proposed to transform facial expression to the two dimensional so-called valence arousal space. With these affective user state parameters available, the course of the interaction can basically be guided smarter, i.e. the computer can provide help to an apparently confused user. However, it has been shown that the valence arousal space transformation can be impaired due to inaccuracies in image based feature extraction. In this article we present an advanced method using image processing and 3-D computer vision technology that on the one hand suppresses this problem through hierarchical analysis. Further, our concept enables the assignment of an intensity level of the affective state, which can be a valuable parameter for the interaction. In this paper we give details on the system concept with the different processing steps and respective results. By the application of our method we achieve improvement of facial expression recognition compared to other state-of-the-art methods. In particular we can distinguish roughly 15 percent more classes while maintaining the high recognition rate.