The study aims at performing an error analysis as well as providing an evaluation of the quality of neural machine translation (NMT) represented by Google Translate when translating relative clauses. The study uses two test suites are composed of sentences that contain relative clauses. The first test suite composes of 108 pair sentences that are translated from English to Arabic whereas the second composes of 72 Arabic sentences that are translated into English. Errors annotation is performed by 6 professional annotators. The study presents a list of the annotated errors divided into accuracy and fluency errors that occur based on MQM. Manual evaluation is also performed by the six professionals along with a BLEU automatic evaluation using the Tilde Me platform. The results show that fluency errors are more frequent than accuracy errors. They also show that the frequency of errors and MT quality when translating from English into Arabic is lower than the frequency of errors and MT quality when translating from Arabic into English is also presented. Based on the performed error analysis and both manual and automatic evaluation, it is pointed out that the gap between MT and professional human translation is still large.

The study focuses on the gender-related issues that face English-Arabic machine translation. It aims to investigate and evaluate gender accuracy in the translations provided by two prominent large language models, Gemini and ChatGPT, recognizing the rich morphological system of Arabic that includes gender marking. The researchers develops a test suite to evaluate gender accuracy in the translation outputs of Gemini and ChatGPT. The evaluation is performed by two professional annotators. That is followed by an analysis of the patterns of the gender-related issues that appear in the translation outputs of the models under study. The results show that Gemini outperformed ChatGPT in almost every aspect when it comes to gender-related translation issues. Both the number of the annotated issues as well as the gender accuracy evaluation came in favor of Gemini. The study introduced different patterns of gender-related translation issues. It also provides recommendations for future research.

This study aims to investigate external feedback strategies and their effects on ChatGPT’s self-correction for the purpose of improving its translation performance. The study evaluates the effectiveness of the proposed strategies and points out which strategy is more effective. To achieve these objectives, the researchers built a test suite that is composed of 200 English sentences. These sentences were translated into Arabic by ChatGPT using a default translation prompt. The translated sentences were manually annotated, and an error marking and an error taxonomy were provided. The researchers, then, selected the 60 sentences with the most errors to be retranslated using self-correction strategies. A professional manual evaluation of the initial translation and the retranslation was performed to evaluate the effectiveness of the used strategies. The study investigated whether the effort spent on error classification is needed to get a better translation when using ChatGPT’s self-correction strategy. The study is very important for translators, post-editors, researchers, and developers of MT, and it provided recommendations for future work.

This study aims to investigate both external and self-provided feedback strategies and their effects on ChatGPT’s self-correction for the purpose of improving its translation performance. The study evaluates the effectiveness of the proposed strategies and points out which strategy is more effective when translating English medical texts into Arabic. The researchers chose 15 English medical texts. These texts were translated into Arabic by ChatGPT using a default translation prompt. The translated texts are manually annotated and evaluated. The researchers, then, retranslate the texts where some of the texts were retranslated using the self-provided feedback and some were retranslated using external feedback. Both feedback strategies are used to exploit ChatGPT self-correction. Manual evaluation of the initial translation and the retranslation are performed to evaluate the effectiveness of the used feedback strategies. The effectiveness of each of these feedback strategies are evaluated based on the improvement rate of the retranslation. The study reaches a conclusion regarding the most efficient feedback strategy, which is error taxonomy feedback strategy. The study is very important for translators, post-editors, researchers, and developers of MT. It also provides recommendations for future work.