An increasing demand for model explainability has accompanied the widespread adoption of transformers in various fields of applications. In this paper, we conduct a survey of the existing literature on the explainability of transformers. We provide a taxonomy of methods based on the combination of transformer components that are leveraged to arrive at the explanation. For each method, we describe its mechanism and survey its applications. We find out that attention-based methods, both alone and in conjunction with activation-based and gradient-based methods, are the most employed ones. A growing attention is also devoted to the deployment of visualization techniques to help the explanation process.
27 February 2016 in Journal of the Science of Food and Agriculture
The problem of understanding and predicting tourist behavior in choosing their destinations is a long-standing one. The first step in the process is to understand users’ intention to visit a country, which may later translate into an actual visit. Would-be tourists may express their intention to visit a destination on social media. Being able to predict their intention may be useful for targeted promotion campaigns. In this paper, we propose an algorithm to predict visit (or revisit) intentions based on the texts in posts on social media. The algorithm relies on a neural network sentence-transformer architecture using optimized embedding and a logistic classifier. Employing two real labeled datasets from Twitter (now X) for training, the algorithm achieved 90% accuracy and balanced performances over the two classes (visit intention vs. no-visit intention). The algorithm was capable of predicting intentions to visit with high accuracy, even when fed with very imbalanced datasets, where the posts showing the intention to visit were an extremely small minority.
6 October 2023 in Digital Scholarship in the Humanities
Automated character identification in movies and TV series has been typically carried out through face detection in video and the association of faces with characters’ names extracted from dialogues or cast lists. We propose a deep learning architecture to identify characters based on subtitles only, precisely through the lexicon those characters employ. The identification task is formalized as a multi-class classification task. We apply our technique to the complete set of episodes in the Gomorrah TV series and achieve an average identification accuracy beyond 94 per cent on the full set of characters.
Moral features are essential components of TV series, helping the audience to engage with the story, exploring themes beyond sheer entertainment, reflecting current social issues, and leaving a long-lasting impact on the viewers. Their presence shows through the language employed in the plot description. Their detection helps regarding understanding the series writers’ underlying message. In this paper, we propose an approach to detect moral features in TV series. We rely on the Moral Foundations Theory (MFT) framework to classify moral features and use the associated MFT dictionary to identify the words expressing those features. Our approach combines that dictionary with word embedding and similarity analysis through a deep learning SBERT (Sentence-Bidirectional Encoder Representations from Transformers) architecture to quantify the comparative prominence of moral features. We validate the approach by applying it to the definition of the MFT moral feature labels as appearing in general authoritative dictionaries. We apply our technique to the summaries of a selection of TV series representative of several genres and relate the results to the actual content of each series, showing the consistency of results.
28 November 2024 in Artificial Intelligence and Law
Consultations are key to gather evidence that informs rulemaking. When analysing the feedback received, it is essential for the regulator to appropriately cluster stakeholders’ opinions, as misclustering may alter the representativeness of the positions, making some of them appear majoritarian when they might not be. The European Commission (EC)’s approach to clustering opinions in consultations lacks a standardized methodology, leading to reduced procedural transparency, while making use of computational tools only sporadically. This paper explores how natural language processing (NLP) technologies may enhance the way opinion clustering is currently conducted by the EC. We examine 830 responses to three legislative proposals (the Artificial Intelligence Act, the Digital Markets Act and the Digital Services Act) using both a lexical and semantic approach. We find that some groups (like small and medium companies) have low similarity across all datasets and methodologies despite being clustered in one opinion group by the EC. The same happens for citizens and consumer associations for the consultation run over the DSA. These results suggest that computational tools actually help reduce misclustering of stakeholders’ opinions and consequently allow greater representativeness of the different positions expressed in consultations. They further suggest that the EC could identify a convergent methodology for all its consultations, where such tools are employed in a consistent and replicable rather than occasionally. Ideally, it should also explain when one methodology is preferred to another. This effort should find its way into the Better Regulation toolbox (EC 2023). Our analysis also paves the way for further research to reach a transparent and consistent methodology for group clustering.