Dr. Maria Chiara Caschera

Fernando Ferri

Emotion Classification from Speech and Text in Videos Using a Multimodal Approach

Fake news detection has gained increasing importance among the research community due to the widespread diffusion of fake news through media platforms. Many dataset have been released in the last few years, aiming to assess the performance of fake news detection methods. In this survey, we systematically review twenty-seven popular datasets for fake news detection by providing insights into the characteristics of each dataset and comparative analysis among them. A fake news detection datasets characterization composed of eleven characteristics extracted from the surveyed datasets is provided, along with a set of requirements for comparing and building new datasets. Due to the ongoing interest in this research topic, the results of the analysis are valuable to many researchers to guide the selection or definition of suitable datasets for evaluating their fake news detection methods.

12 April 2022 in Multimodal Technologies and Interaction

Journal Article

Multimodal interaction systems: information and time features

Emotion classification is a research area in which there has been very intensive literature production concerning natural language processing, multimedia data, semantic knowledge discovery, social network mining, and text and multimedia data mining. This paper addresses the issue of emotion classification and proposes a method for classifying the emotions expressed in multimodal data extracted from videos. The proposed method models multimodal data as a sequence of features extracted from facial expressions, speech, gestures, and text, using a linguistic approach. Each sequence of multimodal data is correctly associated with the emotion by a method that models each emotion using a hidden Markov model. The trained model is evaluated on samples of multimodal sentences associated with seven basic emotions. The experimental results demonstrate a good classification rate for emotions.

1 January 2007 in International Journal of Web and Grid Services

Journal Article

Multimodal Interaction in Gaming

Multimodal interaction systems combine visual information (involving images, text, sketches and so on) with voice, gestures and other modalities to provide flexible and powerful dialogue approaches, enabling users to choose one or more of the multiple interaction modalities. They break down the barriers in adopting mobile devices for value-added services and the use of integrated multiple input modes enables users to benefit from the natural approach used in human communication. This paper deals with the main features of multimodal interaction and systems, starting from the definition of visual language given in Bottoni et al. (1995) and extending it to multimodality. Modal/multimodal message, interpretation and materialisation functions and multimodal sentence are defined. This paper introduces and formally defines the different classes of cooperation between different modes, introducing the time relationships among the involved modalities and the relationships between chunks of information connected with these modalities.

1 January 2013 in Lecture Notes in Computer Science

Book Chapter

Arianna D'Ulizia

InteSe: An Integrated Model for Resolving Ambiguities in Multimodal Sentences

12 September 2012 in IEEE Transactions on Systems, Man, and Cybernetics: Systems

Journal Article

Ambiguities in Sketch-Based Interfaces

The pervasiveness of ambiguity in communication processes suggests addressing the problem of semantic and syntactic ambiguities in multimodal interaction languages. This paper presents an integrated model based on layered, hierarchical, and hidden Markov models for dealing with the complex process of multimodal ambiguity resolution. The proposed model consists of different levels, from the terminals of a multimodal language (terminal elements) to the level of multimodal sentences. A software module implemented the model that has been evaluated in terms of accuracy and robustness. The experimental results show good levels of accuracy and robustness compared with other existing approaches.

1 January 2007

Conference Paper

Danilo Avola