Mathieu De Coster

Mathieu De Coster

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 April 2023 in Universal Access in the Information Society
Verified
Journal Article
Open access
PDF file
MCMathieu De Coster
Mathieu De Coster
DSDimitar Shterionov
Dimitar Shterionov
MHMieke Van Herreweghe
Mieke Van Herreweghe
JDJoni Dambre
Joni Dambre
Automatic translation from signed to spoken languages is an interdisciplinary research domain on the intersection of computer vision, machine translation (MT), and linguistics. While the domain is growing in terms of popularity—the majority of scientific papers on sign language (SL) translation have been published in the past five years—research in this domain is performed mostly by computer scientists in isolation. This article presents an extensive and cross-domain overview of the work on SL translation. We first give a high level introduction to SL linguistics and MT to illustrate the requirements of automatic SL translation. Then, we present a systematic literature review of the state of the art in the domain. Finally, we outline important challenges for future research. We find that significant advances have been made on the shoulders of spoken language MT research. However, current approaches often lack linguistic motivation or are not adapted to the different characteristics of SLs. We explore challenges related to the representation of SL data, the collection of datasets and the evaluation of SL translation models. We advocate for interdisciplinary research and for grounding future research in linguistic analysis of SLs. Furthermore, the inclusion of deaf and hearing end users of SL translation applications in use case identification, data collection, and evaluation, is of utmost importance in the creation of useful SL translation models.

23 April 2022 in Information
Verified
Journal Article
Open access
PDF file
MCMathieu De Coster
Mathieu De Coster
JDJoni Dambre
Joni Dambre
We consider neural sign language translation: machine translation from signed to written languages using encoder–decoder neural networks. Translating sign language videos to written language text is especially complex because of the difference in modality between source and target language and, consequently, the required video processing. At the same time, sign languages are low-resource languages, their datasets dwarfed by those available for written languages. Recent advances in written language processing and success stories of transfer learning raise the question of how pretrained written language models can be leveraged to improve sign language translation. We apply the Frozen Pretrained Transformer (FPT) technique to initialize the encoder, decoder, or both, of a sign language translation model with parts of a pretrained written language model. We observe that the attention patterns transfer in zero-shot to the different modality and, in some experiments, we obtain higher scores (from 18.85 to 21.39 BLEU-4). Especially when gloss annotations are unavailable, FPTs can increase performance on unseen data. However, current models appear to be limited primarily by data quality and only then by data quantity, limiting potential gains with FPTs. Therefore, in further research, we will focus on improving the representations used as inputs to translation models.

4 November 2024
Verified
Conference Paper
Open access
RJRuben Janssens
Ruben Janssens
EVEva Verhelst
Eva Verhelst
MCMathieu De Coster
Mathieu De Coster
Social robots should be able to detect social signals sent by their user, such as when the robot made a mistake or the user feels awkward. As our submission to the ERR@HRI challenge, we present a number of neural and traditional machine learning models to predict when this occurs in a human-robot conversation, based on facial expressions, body pose, and non-verbal speech characteristics. The small size of the dataset, imbalance in the label distribution, and low-frequency label annotations provided significant challenges. However, three of our approaches show promising results: modifying the training of a gated recurrent unit (GRU) model to predict at lower frequency than that of the input features, using an embedding layer and convolutional neural network to pre-process temporal data before feeding it to the GRU, and using traditional random forests.

25 January 2024 in Universal Access in the Information Society
Verified
Journal Article
Open access
PDF file
MCMathieu De Coster
Mathieu De Coster
DSDimitar Shterionov
Dimitar Shterionov
MHMieke Van Herreweghe
Mieke Van Herreweghe
JDJoni Dambre
Joni Dambre

26 July 2024
Verified
Book Chapter
Closed access
PDF file
VVVincent Vandeghinste
Vincent Vandeghinste
MSMirella De Sisto
Mirella De Sisto
SGSantiago Egea Gómez
Santiago Egea Gómez
MCMathieu De Coster
Mathieu De Coster

12 November 2024
Verified
Book Chapter
Closed access
PDF file
VVVincent Vandeghinste
Vincent Vandeghinste
MSMirella De Sisto
Mirella De Sisto
MKMaria Kopf
Maria Kopf
MSMarc Schulder
Marc Schulder
CBCaro Brosens
Caro Brosens
MCMathieu De Coster
Mathieu De Coster