31 March 2023 in ACM Transactions on Asian and Low-Resource Language Information Processing
Deep Learning is one of the most promising technologies compared to other methods in the context of machine translation. It has been proven to achieve impressive results on large amounts of parallel data for well-endowed languages. Nevertheless, for low-resource languages such as the Arabic Dialects, Deep Learning models failed due to the lack of available parallel corpora. In this article, we present a method to create a parallel corpus to build an effective NMT model able to translate into MSA, Tunisian Dialect texts present in social networks. For this, we propose a set of data augmentation methods aiming to increase the size of the state-of-the-art parallel corpus. By evaluating the impact of this step, we noticed that it has effectively boosted both the size and the quality of the corpus. Then, using the resulted corpus, we compare the effectiveness of CNN, RNN and transformers models to translate Tunisian Dialect into MSA. Experiments show that a better translation is achieved by the transformer model with a BLEU score of 60 vs., respectively, 33.36 and 53.98 with RNN and CNN models.
10 December 2024 in Journal of Business Analytics
17 April 2025 in IEEE Transactions on Artificial Intelligence
Named Entity Recognition (NER) is among the main tasks of Natural Language Processing (NLP). NER is a critical and fundamental component for several NLP applications including Information Retrieval (IR), Question-Answering (QA) and Machine Translation (MT). While several NER models for formal languages such as English and Modern Standard Arabic (MSA) have emerged, Arabic dialects remain in their infancy. We noted that the most recent researches focused on the study of a single Arabic dialect, hence the absence of a perfect multidialectal NER model. In this paper, we present the ARDIAL-BERT, the first multidialectal NER model which was built upon a continuous pretraining and then a finetuning on Arabic dialect publicly available datasets grouped by region (Levantine, Maghrebi, Egyptian and Gulf). The model was built upon the last updated version of BERT transfer transformers after several experiments on various BERT NER models. We approached our contribution on two different tasks: first we built ARDIAL-NER, an Arabic multidialect dataset extracted from existing NER datasets. ARDIAL-NER was manually annotated and contains a total of 53,539 entities of 369,372 tokens composing 21683 sentences. Second, we conducted a continual pretraining process using additional unannotated data, and then we guided a finetuning on new annotated NER datasets. The continuous learning system can be applied at different levels: updating model parameters, incorporating new language data, and training on new labels. Our results demonstrate its effectiveness to varying degrees. Our approach showed that it exhibited a greater ability by achieving superior results compared with both baselines and previous models. This demonstrates the capabilities of grouping Arabic dialects by region and the good selection of data that matches well with the baseline transformers.
2 December 2023 in Language Resources and Evaluation