Eva Vanmassenhove

Dr. Eva Vanmassenhove

Tilburg School of Humanities and Digital Sciences, Department Cognitive Science and Artificial Intelligence, Tilburg, The Netherlands

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 February 2023
Verified
Book Chapter
Closed access
PDF file
DSDimitar Shterionov
Dimitar Shterionov
EVEva Vanmassenhove
Eva Vanmassenhove

31 August 2021 in arXiv
Verified
Preprint
Open access
PDF file
EVEva Vanmassenhove
Eva Vanmassenhove
JMJohanna Monti
Johanna Monti
Languages differ in terms of the absence or presence of gender features, the number of gender classes and whether and where gender features are explicitly marked. These cross-linguistic differences can lead to ambiguities that are difficult to resolve, especially for sentence-level MT systems. The identification of ambiguity and its subsequent resolution is a challenging task for which currently there aren't any specific resources or challenge sets available. In this paper, we introduce gENder-IT, an English--Italian challenge set focusing on the resolution of natural gender phenomena by providing word-level gender tags on the English source side and multiple gender alternative translations, where needed, on the Italian target side.

13 May 2025 in arXiv
Verified
Preprint
Open access
PDF file
CMChiara Manna
Chiara Manna
AAAfra Alishahi
Afra Alishahi
FBFrédéric Blain
Frédéric Blain
EVEva Vanmassenhove
Eva Vanmassenhove
While gender bias in modern Neural Machine Translation (NMT) systems has received much attention, traditional evaluation metrics do not to fully capture the extent to which these systems integrate contextual gender cues. We propose a novel evaluation metric called Minimal Pair Accuracy (MPA), which measures the reliance of models on gender cues for gender disambiguation. MPA is designed to go beyond surface-level gender accuracy metrics by focusing on whether models adapt to gender cues in minimal pairs -- sentence pairs that differ solely in the gendered pronoun, namely the explicit indicator of the target's entity gender in the source language (EN). We evaluate a number of NMT models on the English-Italian (EN--IT) language pair using this metric, we show that they ignore available gender cues in most cases in favor of (statistical) stereotypical gender interpretation. We further show that in anti-stereotypical cases, these models tend to more consistently take masculine gender cues into account while ignoring the feminine cues. Furthermore, we analyze the attention head weights in the encoder component and show that while all models encode gender information to some extent, masculine cues elicit a more diffused response compared to the more concentrated and specialized responses to feminine gender cues.