This article introduces Instrução Assistida com Inteligência Artificial (INACIA), a groundbreaking system designed to integrate Large Language Models (LLMs) into the operational framework of Brazilian Federal Court of Accounts. The system automates various stages of case analysis, including basic information extraction, admissibility examination, Periculum in mora and Fumus boni iuris analyses, and recommendations generation. Through a series of experiments, we demonstrate INACIA’s potential in extracting relevant information from case documents, evaluating its legal plausibility, and formulating propositions for judicial decision-making. Utilizing a validation dataset alongside LLMs, our evaluation methodology, to the best of our knowledge, presents a novel approach to assessing system performance, correlating highly with human judgment. These results underscore INACIA’s potential in complex legal task handling while also acknowledging the current limitations. This study discusses possible improvements and the broader implications of applying Artificial Intelligence (AI) in legal contexts, suggesting that INACIA represents a significant step toward integrating AI in legal systems globally, albeit with cautious optimism grounded in the empirical findings.
We present Quati, a dataset specifically designed for evaluating Information Retrieval (IR) systems for the Brazilian Portuguese language. It comprises a collection of queries formulated by native speakers and a curated set of documents sourced from a selection of frequently accessed Brazilian Portuguese websites, which ensures a representative and relevant corpus. To label the query–document pairs, we use a state-of-the-art LLM, which shows inter-annotator agreement levels comparable to human performance in our assessments. Our annotation methodology is described, enabling the cost-effective creation of similar datasets for other languages, with an arbitrary number of labeled documents per query. As a baseline, we evaluate a diverse range of open-source and commercial retrievers. Quati is publicly available at https://huggingface.co/datasets/unicamp-dl/quati, and all scripts at https://github.com/unicamp-dl/quati.
Individuals with complex communication needs often rely on augmentative and alternative communication (AAC) systems to have conversations and communicate their wants. Such systems allow message authoring by arranging pictograms in sequence. However, the difficulty of finding the desired item to complete a sentence can increase as the user’s vocabulary increases. This paper proposes using BERTimbau, a Brazilian Portuguese version of Bidirectional Encoder Representations from Transformers (BERT), for pictogram prediction in AAC systems. To fine-tune BERTimbau, we constructed an AAC corpus for Brazilian Portuguese to use as a training corpus. We tested different approaches to representing a pictogram for prediction: as a word (using pictogram captions), as a concept (using a dictionary definition), and as a set of synonyms (using related terms). We also evaluated the usage of images for pictogram prediction. The results demonstrate that using embeddings computed from the pictograms’ caption, synonyms, or definitions have a similar performance. Using synonyms leads to lower perplexity, but using captions leads to the highest accuracies. This paper provides insight into how to represent a pictogram for prediction using a BERT-like model and the potential of using images for pictogram prediction.