In this paper we present a study on music mood classification using audio and lyrics information. The mood of a song is expressed by means of musical features but a relevant part also seems to be conveyed by the lyrics. We evaluate each factor independently and explore the possibility to combine both, using natural language processing and music information retrieval techniques. We show that standard distance-based methods and latent semantic analysis are able to classify the lyrics significantly better than random, but the performance is still quite inferior to that of audio-based techniques. We then introduce a method based on differences between language models that gives performances closer to audio-based classifiers. Moreover, integrating this in a multimodal system (audio+text) allows an improvement in the overall performance. We demonstrate that lyrics and audio information are complementary, and can be combined to improve a classification system.

Stress can be considered a mental/physiological reaction in conditions of high discomfort and challenging situations. The levels of stress can be reflected in both the physiological responses and speech signals of a person. Therefore the study of the fusion of the two modalities is of great interest. For this cause, public datasets are necessary so that the different proposed solutions can be comparable. In this work, a publicly available multimodal dataset for stress detection is introduced, including physiological signals and speech cues data. The physiological signals include electrocardiograph (ECG), respiration (RSP), and inertial measurement unit (IMU) sensors equipped in a smart vest. A data collection protocol was introduced to receive physiological and audio data based on alterations between well-known stressors and relaxation moments. Five subjects participated in the data collection, where both their physiological and audio signals were recorded by utilizing the developed smart vest and audio recording application. In addition, an analysis of the data and a decision-level fusion scheme is proposed. The analysis of physiological signals includes a massive feature extraction along with various fusion and feature selection methods. The audio analysis comprises a state-of-the-art feature extraction fed to a classifier to predict stress levels. Results from the analysis of audio and physiological signals are fused at a decision level for the final stress level detection, utilizing a machine learning algorithm. The whole framework was also tested in a real-life pilot scenario of disaster management, where users were acting as first responders while their stress was monitored in real time.

While collaborative filtering often yields very good recommendation results, in many real-world recommendation scenarios cold-start and data sparseness remain important problems. This paper presents a hybrid recommender system that integrates user demographics and item characteristics, around a collaborative filtering core based on user-item interactions. The recommender system is evaluated on Movie lens data (including genre information and user data) as well as real-world data from a discount coupon provider. We show that the inclusion of additional item and user information can have great impact on recommendation quality, especially in settings where little interaction data is available.

Flood management and media production planning are both tasks that require timely and sound decision making, as well as effective collaboration between professionals in a team split between remote headquarter operators and in situ actors. This paper presents an extended reality (XR) platform that utilizes interactive and immersive technologies and integrates artificial intelligence (AI) algorithms to support the professionals and the public involved in such incidents and events. The developed XR tools address various specialized end-user needs of different target groups and are fueled by modules that intelligently collect, analyze, and link data from heterogeneous sources while considering user-generated content. This platform was tested in a flood-prone area and in a documentary planning scenario, where it was used to create immersive and interactive experiences. The findings demonstrate that it increases situation awareness and improves the overall performance of the professionals involved. The proposed XR system represents an innovative technological approach for tackling the challenges of flood management and media production, one that also has the potential to be applied in other fields.