Martin Tutek

Martin Tutek

Auto-generated profile, based on publicly available information

Martin Tutek

UKP Lab, Technical University of Darmstadt

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 April 2022 in IEEE Access
Verified
Journal Article
Open access
MTMartin Tutek
Martin Tutek
JSJan Snajder
Jan Snajder
Natural language processing (NLP) has been one of the subfields of artificial intelligence much affected by the recent neural revolution. Architectures such as recurrent neural networks (RNNs) and attention-based transformers helped propel the state of the art across various NLP tasks, such as sequence classification, machine translation, and natural language inference. However, if neural models are to be used in high-stakes decision making scenarios, the explainability of their decisions becomes a paramount issue. The attention mechanism has offered some transparency in the workings of otherwise black-box RNN models: attention weights (scalar values assigned input words) invite to be interpreted as the importance of that word, providing a simple method of interpretability. Recent work, however, has questioned the faithfulness of this practice. Subsequent experiments have shown that faithfulness of attention weights may still be achieved by incorporating word-level objectives in the training process of neural networks. In this article, we present a study that extends the techniques for improving faithfulness of attention based on regularization methods that promote retention of word-level information. We perform extensive experiments on a wide array of recurrent neural architectures and analyze to what extent the explanations provided by inspecting attention weights are correlated with the human notion of importance. We find that incorporating tying regularization consistently improves both the faithfulness (−0.14 F1, +0.07 Brier, on average) and plausibility ( $+53.6\%$ attention mass on salient tokens) of explanations obtained through inspecting attention weights across analyzed datasets and models.

1 January 2023
Verified
Conference Paper
Open access
PDF file
JJJosip Jukić
Josip Jukić
MTMartin Tutek
Martin Tutek
JSJan Snajder
Jan Snajder
Josip Jukić, Martin Tutek, Jan Snajder. Findings of the Association for Computational Linguistics: ACL 2023. 2023.