Developing tools to understand and visualize lifestyle is of high interest when addressing the improvement of habits and well-being of people. Routine, defined as the usual things that a person does daily, helps describe the individuals’ lifestyle. With this paper, we are the first ones to address the development of novel tools for automatic discovery of routine days of an individual from his/her egocentric images. In the proposed model, sequences of images are firstly characterized by semantic labels detected by pre-trained CNNs. Then, these features are organized in temporal-semantic documents to later be embedded into a topic models space. Finally, Dynamic-Time-Warping and Spectral-Clustering methods are used for final day routine/non-routine discrimination. Moreover, we introduce a new EgoRoutine-dataset, a collection of 104 egocentric days with more than 100.000 images recorded by 7 users. Results show that routine can be discovered and behavioural patterns can be observed.

Recent studies have shown that the environment where people eat can affect their nutritional behaviour. In this work, we provide automatic tools for personalised analysis of a person's health habits by the examination of daily recorded egocentric photo-streams. Specifically, we propose a new automatic approach for the classification of food-related environments, that is able to classify up to 15 such scenes. In this way, people can monitor the context around their food intake in order to get an objective insight into their daily eating routine. We propose a model that classifies food-related scenes organized in a semantic hierarchy. Additionally, we present and make available a new egocentric dataset composed of more than 33000 images recorded by a wearable camera, over which our proposed model has been tested. Our approach obtains an accuracy and F-score of 56% and 65%, respectively, clearly outperforming the baseline methods.

Emotion recognition is the task of identifying and understanding human emotions from data. In the field of computer vision, there is a growing interest due to the wide range of possible applications in smart cities, health, marketing, and surveillance, among others. To date, several datasets have been proposed to allow techniques to be trained, validated, and finally deployed to production. However, these techniques have several limitations related to the construction of these datasets. In this work, we survey the datasets currently employed in state-of-the-art emotion recognition, to list and discuss their applicability and limitations in real-world scenarios. We propose experiments on the data to extract essential insights related to the provided visual information in each dataset and discuss how they impact the training and validation of techniques. We also investigate the presence of nonverbal cues in the datasets and propose experiments regarding their representativeness, visibility, and data quality. Among other discussions, we show that EMOTIC has more diverse context representations than CAER, however, with conflicting annotations. Finally, we discuss application scenarios and how techniques to approach them could leverage these datasets, suggesting approaches based on findings from these datasets to help guide future research and deployment. With this work we expect to provide a roadmap for upcoming research and experimentation in emotion recognition under real-world conditions.