Natural language processing (NLP) tasks can be addressed with several deep learning architectures, and many different approaches have proven to be efficient. This study aims to briefly summarize the use cases for NLP tasks along with the main architectures. This research presents transformer-based solutions for NLP tasks such as Bidirectional Encoder Representations from Transformers (BERT), and Generative Pre-Training (GPT) architectures. To achieve that, we conducted a step-by-step process in the review strategy: identify the recent studies that include Transformers, apply filters to extract the most consistent studies, identify and define inclusion and exclusion criteria, assess the strategy proposed in each study, and finally discuss the methods and architectures presented in the resulting articles. These steps facilitated the systematic summarization and comparative analysis of NLP applications based on Transformer architectures. The primary focus is the current state of the NLP domain, particularly regarding its applications, language models, and data set types. The results provide insights into the challenges encountered in this research domain.

This paper aims to investigate the use of a Romanian data source, different classifiers, and text data augmentation techniques to implement a fake news detection system. The paper focusses on text data augmentation techniques to improve the efficiency of fake news detection tasks. This study provides two approaches for fake news detection based on content and context features found in the Factual.ro data set. For this purpose, we implemented two data augmentation techniques, Back Translation (BT) and Easy Data Augmentation (EDA), to improve the performance of the models. The results indicate that the implementation of the BT and EDA techniques successfully improved the performance of the classifiers used in our study. The results of our content-based approach show that an Extra Trees Classifier model is the most effective, whether data augmentation is used or not, as it produced the highest accuracy, precision, F1 score, and Kappa. The Random Forest Classifier with BT yielded the best results of the context-based experiment overall, with the highest accuracy, recall, F1 score, and Kappa. Furthermore, we found that BT and EDA led to an increase in the AUC scores of all models in both content-based and context-based data sets.

Anomalies and fake data can be identified by applying supervised learning algorithms to news sources. These techniques can help reduce the negative impact of fake news on consumers. Fake news is a widespread problem around the world and is also gaining momentum in Romania. For the detection of fake news in Romanian, we investigate methods to construct a Romanian data set and apply algorithms that offer the highest performance. To improve performance, we use a data augmentation technique called back translation in conjunction with the Support Vector Machine classifier.