Este trabalho apresenta estratégias para Avaliação Automática de Redações (AAR) escritas em português por meio de uma abordagem baseada na definição de features e modelos de AAR específicos para cada competência da matriz de referência do ENEM. Foram investigados métodos baseados em engenharia de features, embeddings e Redes Neurais Recorrentes. Apesar dos resultados obtidos serem melhores do que trabalhos relacionados, novos estudos devem ser conduzidos a fim de melhorar o desempenho dos modelos de AAR para a língua portuguesa.
The proliferation of toxic messages on the Web has led many social networks to limit or shut down user comments, as these messages are harmful to people and keep them away from online platforms. Most approaches deal with this unacceptable language form, focusing only on identifying if a comment is toxic or not, using a supervised learning strategy, leaving aside the various types of harmful messages, such as LGBT+phobia, racism, xenophobia, and others. In this paper, we investigate three contextual word embedding models: BERTimbau, BERTweet.BR, and LLaMA 3.1 within a semi-supervised approach to distinguish whether a comment is toxic. Also, we explore this strategy to detect six types of toxicity: LGBT+phobia, insult, racism, obscenity, xenophobia, and misogyny. This task is defined as a multi-label classification problem, as a comment may contain several types of abusive language. We evaluated our approach using the ToLD-BR corpus and achieved competitive results for the binary toxicity classification task. In the context of multi-label toxicity detection, our best result outperformed approaches based on supervised learning, using significantly fewer labeled data, and emphasized their efficiency and practicality.
This paper analyzed the robustness of a state-of-the-art Automated Essay Scoring (AES) model by applying various linguistically motivated perturbations to the Essay-BR corpus. Our findings reveal that the AES model failed to detect these adversarial modifications, often assigning higher scores to the disturbed essays than to the original ones.
Artificial Intelligence (AI) models, including deep learning and rule-based approaches, often function as black boxes, limiting transparency and increasing uncertainty in decisions. This study addresses spurious correlations, defined as associations between patterns and classes that do not reflect causal relationships, affecting AI models’ reliability and applicability. In Natural Language Processing (NLP), these correlations lead to inaccurate predictions, biases, and challenges in model generalization. We propose a method that employs Explainable Artificial Intelligence (XAI) techniques to detect spurious patterns in textual datasets for binary classification tasks. The method applies the K-means algorithm to cluster patterns and interprets them based on their distance from centroids. It hypothesizes that patterns farther from the centroids are more likely to be spurious than those closer to them. We apply the method to public procurement datasets from the Court of Auditors of Piauí (TCE-PI) using models based on Support Vector Machine (SVM) and Logistic Regression with text representations from TFIDF and Word Embeddings, as well as a BERT model. The analysis is extended to the IMDB movie review dataset to evaluate generalizability. The results support the hypothesis that patterns farther from centroids exhibit higher spuriousness potential and demonstrate the clustering’s consistency across models and datasets. The method operates independently of the techniques used in its stages, enabling the automatic detection and quantification of spurious patterns without prior human intervention.