Grammatical error correction (GEC) is one of the major tasks in natural language processing (NLP) which has recently attracted great attention from researchers. The performance of universal languages such as English and Chinese in the GEC system has improved significantly. This could be attributed to the large number of powerful applications supported by neural network models and pretrained language models. Referring to the satisfactory results of the universal language in the GEC task and the lack of research on the GEC task for low-resource languages, especially Indonesian, this paper proposes an automatic model for Indonesian grammar correction based on the Transformer architecture which can be applied to other low-resource language texts. Furthermore, we build a large corpus of the Indonesian language that can be utilized for evaluating the next Indonesian GEC task. We evaluate the models in this dataset, and the results show that the Transformer-based automatic error correction model achieved significant and satisfactory results compared with the results of previous research models.
26 November 2024 in Journal of Network and Systems Management
Sentiment Analysis (SA) is one of the fast-growing research tasks in Natural Language Processing (NLP), which aims to identify the attitude of online users or communities regarding to a specific topic, service, or product. It helps to evaluate the overall tonality of a document to make a good decision in the right direction. Arabic SA still growing slowly compared to English and Chinese languages because of some challenges including lack of corpora, Arabic language complexity, and the spread of many local Arabic dialects. The current study presents a work-in-progress for Arabic SA based on capsule network and Google BERT multi-languages.BERT Google language model achieved state-of-the-art results for several NLP tasks, that aim to pre-train deep bidirectional (right and left) representations on the context in all layers. Recently, capsule networks become one of the most successful techniques in image processing and NLP, it is consists of a group of neurons that represents different features of the same entity. Each layer in a capsule network contains many capsules that encode spatial information and the likelihood of existing input data. Not like traditional neural techniques, capsule networks allow the model to capture the likeliness of each feature and its variants, which affects positively the quality of the final predicted features. The proposed model was trained based on a small dataset. Although, the results were encouraging and competitive compared to the state-of-the-art Arabic SA models.
The correction of grammatical errors in natural language processing is a crucial task as it aims to enhance the accuracy and intelligibility of written language. However, developing a grammatical error correction (GEC) framework for low-resource languages presents significant challenges due to the lack of available training data. This article proposes a novel GEC framework for low-resource languages, using Arabic as a case study. To generate more training data, we propose a semi-supervised confusion method called the equal distribution of synthetic errors (EDSE), which generates a wide range of parallel training data. Additionally, this article addresses two limitations of the classical seq2seq GEC model, which are unbalanced outputs due to the unidirectional decoder and exposure bias during inference. To overcome these limitations, we apply a knowledge distillation technique from neural machine translation. This method utilizes two decoders, a forward decoder right-to-left and a backward decoder left-to-right, and measures their agreement using Kullback-Leibler divergence as a regularization term. The experimental results on two benchmarks demonstrate that our proposed framework outperforms the Transformer baseline and two widely used bidirectional decoding techniques, namely asynchronous and synchronous bidirectional decoding. Furthermore, the proposed framework reported the highest F1 score, and generating synthetic data using the equal distribution technique for syntactic errors resulted in a significant improvement in performance. These findings demonstrate the effectiveness of the proposed framework for improving grammatical error correction for low-resource languages, particularly for the Arabic language.
22 April 2026 in Biomedical Signal Processing and Control