Crowds express emotions as a collective individual, which is evident from the sounds that a crowd produces in particular events, e.g., collective booing, laughing or cheering in sports matches, movies, theaters, concerts, political demonstrations, and riots. A critical question concerning the innovative concept of crowd emotions is whether the emotional content of crowd sounds can be characterized by frequency-amplitude features, using analysis techniques similar to those applied on individual voices, where deep learning classification is applied to spectrogram images derived by sound transformations. In this work, we present a technique based on the generation of sound spectrograms from fragments of fixed length, extracted from original audio clips recorded in high-attendance events, where the crowd acts as a collective individual. Transfer learning techniques are used on a convolutional neural network, pre-trained on low-level features using the well-known ImageNet extensive dataset of visual knowledge. The original sound clips are filtered and normalized in amplitude for a correct spectrogram generation, on which we fine-tune the domain-specific features. Experiments held on the finally trained Convolutional Neural Network show promising performances of the proposed model to classify the emotions of the crowd.

This work concludes the first study on mouth-based emotion recognition while adopting a transfer learning approach. Transfer learning results are paramount for mouth-based emotion emotion recognition, because few datasets are available, and most of them include emotional expressions simulated by actors, instead of adopting real-world categorisation. Using transfer learning, we can use fewer training data than training a whole network from scratch, and thus more efficiently fine-tune the network with emotional data and improve the convolutional neural network’s performance accuracy in the desired domain. The proposed approach aims at improving emotion recognition dynamically, taking into account not only new scenarios but also modified situations to the initial training phase, because the image of the mouth can be available even when the whole face is visible only in an unfavourable perspective. Typical applications include automated supervision of bedridden critical patients in a healthcare management environment, and portable applications supporting disabled users having difficulties in seeing or recognising facial emotions. This achievement takes advantage of previous preliminary works on mouth-based emotion recognition using deep-learning, and has the further benefit of having been tested and compared to a set of other networks using an extensive dataset for face-based emotion recognition, well known in the literature. The accuracy of mouth-based emotion recognition was also compared to the corresponding full-face emotion recognition; we found that the loss in accuracy is mostly compensated by consistent performance in the visual emotion recognition domain. We can, therefore, state that our method proves the importance of mouth detection in the complex process of emotion recognition.

In this project we propose a new approach for emotion recognition using web-based similarity (e.g. confidence, PMI and PMING). We aim to extract basic emotions from short sentences with emotional content (e.g. news titles, tweets, captions), performing a web-based quantitative evaluation of semantic proximity between each word of the analyzed sentence and each emotion of a psychological model (e.g. Plutchik, Ekman, Lovheim). The phases of the extraction include: text preprocessing (tokenization, stop words, filtering), search engine automated query, HTML parsing of results (i.e. scraping), estimation of semantic proximity, ranking of emotions according to proximity measures. The main idea is that, since it is possible to generalize semantic similarity under the assumption that similar concepts co-occur in documents indexed in search engines, therefore also emotions can be generalized in the same way, through tags or terms that express them in a particular language, ranking emotions. Training results are compared to human evaluation, then additional comparative tests on results are performed, both for the global ranking correlation (e.g. Kendall, Spearman, Pearson) both for the evaluation of the emotion linked to each single word. Different from sentiment analysis, our approach works at a deeper level of abstraction, aiming to recognize specific emotions and not only the positive/negative sentiment, in order to predict emotions as semantic data.

 Humans react to animal emotions, and animals react to human emotions because we share similar emotional and neurological mirroring systems. Mirror neurons fire both when an animal performs an action and when the animal observes the same action performed by another individual. This neurological system has been linked to social behaviors and abilities, from empathy to learning by imitation, both in intra-species and in inter-species communications. The aim of this paper is to study if a machine learning system can recognize animal emotions, starting from dogs' basic emotions of joy and anger, and to investigate the opportunity of future applications concerning systems of prosthetic knowledge to help people without the proper experience or capability to understand animals aggressivity or friendliness, or for supportive systems in Artificial Intelligence.

In this work, we present SEMO, a Semantic Model for Emotion Recognition, which enables users to detect and quantify the emotional load related to basic emotions hidden in short, emotionally rich sentences (e.g. news titles, tweets, captions). The idea of assessing the semantic similarity of concepts by looking at the occurrences and co-occurrences of terms describing them in pages indexed by a search engine can be directly extended to emotions, and to the words expressing them in different languages. The emotional content associated to a particular emotion for a term can thus be estimated using web-based similarity measures, e.g. Confidence, PMI, NGD and PMING, aggregating the distance computed by a model of emotions, e.g. Ekman, Plutchik and Lovheim. Emotions are ranked based on their similarity to the analyzed text, describing each sentence through a vector of values of emotion load, which form the Vector Space Model for the chosen emotion model and similarity measures. The model is tested comparing experimental results to a ground truth in literature. SEMO takes care of both the phases of data collection and data analysis, to produce knowledge to be used in application domains such as social robots, recommender systems, and human-machine interactive systems