The availability of large annotated corpora from social media and the development of powerful classification approaches have contributed in an unprecedented way to tackle the challenge of monitoring users’ opinions and sentiments in online social platforms across time. Such linguistic data are strongly affected by events and topic discourse, and this aspect is crucial when detecting phenomena such as hate speech, especially from a diachronic perspective. We address this challenge by focusing on a real case study: the “Contro l’odio” platform for monitoring hate speech against immigrants in the Italian Twittersphere. We explored the temporal robustness of a BERT model for Italian (AlBERTo), the current benchmark on non-diachronic detection settings. We tested different training strategies to evaluate how the classification performance is affected by adding more data temporally distant from the test set and hence potentially different in terms of topic and language use. Our analysis points out the limits that a supervised classification model encounters on data that are heavily influenced by events. Our results show how AlBERTo is highly sensitive to the temporal distance of the fine-tuning set. However, with an adequate time window, the performance increases, while requiring less annotated data than a traditional classifier.
The presence and integration of immigrants is one of the most controversial issues in our society, and given current worldwide political instabilities, it will likely become ever more prominent in the cultural and political debate. Social media play an increasingly important role in how citizens debate opinions and react to local and global events. However, several studies point out the danger of social media as a breeding ground for online hate speech (or cyberhate). We propose a novel approach to the exploratory analysis of social phenomena based on the integration of automatic detection of cyberhate against immigrants with offline indicators. We gathered data from the Italian Twittersphere and from the main supplier of official statistical data in Italy (ISTAT). We developed a supervised classification model for hate speech detection, trained on a corpus of Italian tweets manually annotated for hate speech against immigrants, and use it to automatically annotate a large sample of geo-tagged tweets over a span of six years. We crossed this data with the ISTAT data, exploring three macro-indicators related to employment, education and crime. We found correlations suggesting an interplay between economical and cultural factors and the expression of hate online.