Research in the field of translation studies suggests that translation product features can indicate translation difficulty. In the current pilot study, we investigate three of these features, namely the number of errors made in a translation, word translation entropy and degree of syntactic equivalence. We correlate these translation product features with translation process features that can be put together into three categories: duration, revision and gaze information. These features serve as a proxy for the cognitive effort required to solve difficulties in translation. The data that we used was gathered from professional translators as well as students of translation studies. The product data contains manual error annotations of the translations, automatically calculated entropy values and syntactic re-ordering metrics. The process data are derived from keystroke and eye-tracking data gathered during the translation process. By correlating product and process data, we inspect how translation difficulty is reflected in the translation process and whether it is feasible to use product features to predict difficulties in translation. In addition, we also compare data of professional translators and students. We will show that correlations between process and product features exist, which opens many doors to further research on translatability.
We identify a number of aspects that can boost the performance of Neural Fuzzy Repair (NFR), an easy-to-implement method to integrate translation memory matches and neural machine translation (NMT). We explore various ways of maximising the added value of retrieved matches within the NFR paradigm for eight language combinations, using Transformer NMT systems. In particular, we test the impact of different fuzzy matching techniques, sub-word-level segmentation methods and alignment-based features on overall translation quality. Furthermore, we propose a fuzzy match combination technique that aims to maximise the coverage of source words. This is supplemented with an analysis of how translation quality is affected by input sentence length and fuzzy match score. The results show that applying a combination of the tested modifications leads to a significant increase in estimated translation quality over all baselines for all language combinations.
Characteristics of the translation product are often used in translation process research as predictors for cognitive load, and by extension translation difficulty. In the last decade, user-activity information such as eye-tracking data has been increasingly employed as an experimental tool for that purpose. In this paper, we take a similar approach. We look for significant effects that different predictors may have on three different eye-tracking measures: First Fixation Duration (duration of first fixation on a token), Eye-Key Span (duration between first fixation on a token and the first keystroke contributing to its translation), and Total Reading Time on source tokens (sum of fixations on a token). As predictors we make use of a set of established metrics involving (lexico)semantics and word order, while also investigating the effect of more recent ones concerning syntax, semantics or both. Our results show a, particularly late, positive effect of many of the proposed predictors, suggesting that both fine-grained metrics of syntactic phenomena (such as word reordering) as well as coarse-grained ones (encapsulating both syntactic and semantic information) contribute to translation difficulties. The effect on especially late measures may indicate that the linguistic phenomena that our metrics capture (e.g., word reordering) are resolved in later stages during cognitive processing such as problem-solving and revision.
11 April 2023 in International Journal of Learner Corpus Research
This article analyses the extent to which four well-known general cognitive constraints – syntactic priming, cognitive routinisation, markedness of coding and structural integration – impact the linguistic output of translation students and professional translators similarly. It takes subject placement variation in Dutch as a test case to gauge the effect of the four constraints and relies on a controlled corpus of student and professional French-to-Dutch L1 news translations, from which all declarative main clauses with either a preverbal or a postverbal subject were extracted. All corpus instances were annotated for four random variables, the fixed variable expertise and ten other fixed variables, which were considered good proxies for the cognitive constraints. A mixed-effects regression analysis reveals that by and large the cognitive constraints have an identical effect on student and professional translators’ output, with priming and structural integration having the strongest impact on subject placement. However, students diverge from professionals when translating French clauses with a left-dislocated adjunct into Dutch, which is interpreted as an indication of a difference in automatisation when dealing with specific French-Dutch cross-linguistic differences.
This paper introduces Fietje, a family of small language models (SLMs) specifically designed for the Dutch language. The model is based on Phi 2, an English-centric model of 2.7 billion parameters. Fietje demonstrated competitive results with larger language models upon its release. A core emphasis of this work is transparency and reproducibility: Fietje is fully open-source, with model weights, datasets, training, and evaluation code all publicly accessible. The paper discusses the performance of Fietje and many other models on an extensive evaluation suite of benchmarks on reasoning, sentiment analysis, world knowledge, linguistic acceptability and word sense disambiguation. Evaluation results illustrate the rapid progress in the field of LLMs, where recent small models outperform older, larger models that were fine-tuned for Dutch. This trend signals an exciting future for Dutch language processing, suggesting that even compact LLMs are becoming increasingly capable. Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to enhance these models even further, broadening their applicability and accessibility. Fietje is only an intermediate step in improving accessibility to language technology for users of the Dutch language.