Leonardo Rocha

Leonardo Rocha

Auto-generated profile, based on publicly available information

Leonardo Rocha

Universidade Federal de São João del Rei (UFSJ)

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 February 2022 in Expert Systems with Applications
Verified
Journal Article
Closed access
NSNícollas Silva
Nícollas Silva
HWHeitor Werneck
Heitor Werneck
TSThiago Silva
Thiago Silva
APAdriano C.M. Pereira
Adriano C.M. Pereira
LRLeonardo Rocha
Leonardo Rocha

1 May 2021 in Information Processing & Management
Verified
Journal Article
Closed access
WCWashington Cunha
Washington Cunha
VMVítor Mangaravite
Vítor Mangaravite
CGChristian Gomes
Christian Gomes
SCSérgio Canuto
Sérgio Canuto
ERElaine Resende
Elaine Resende
MGMarcos André Gonçalves
Marcos André Gonçalves

13 July 2023 in ACM Computing Surveys
Verified
Journal Article
Open access
WCWashington Cunha
Washington Cunha
FVFelipe Viegas
Felipe Viegas
CFCelso França
Celso França
TRThierson Rosa
Thierson Rosa
LRLeonardo Rocha
Leonardo Rocha
MGMarcos André Gonçalves
Marcos André Gonçalves
Progress in natural language processing has been dictated by the rule of more : more data, more computing power, more complexity, best exemplified by deep learning Transformers. However, training (or fine-tuning) large dense models for specific applications usually requires significant amounts of computing resources. One way to ameliorate this problem is through data engineering instead of the algorithmic or hardware perspectives. Our focus here is an under-investigated data engineering technique, with enormous potential in the current scenario – Instance Selection (IS) (a.k.a. Selective Sampling, Prototype Selection). The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining or improving the effectiveness (accuracy) of the trained models and reducing the training process cost. We survey classical and recent state-of-the-art IS techniques and provide a scientifically sound comparison of IS methods applied to an essential natural language processing task—Automatic Text Classification (ATC). IS methods have been normally applied to small tabular datasets and have not been systematically compared in ATC. We consider several neural and non-neural state-of-the-art ATC solutions and many datasets. We answer several research questions based on tradeoffs induced by a tripod (training set reduction, effectiveness, and efficiency). Our answers reveal an enormous unfulfilled potential for IS solutions. Specially, we show that in 12 out of 19 datasets, specific IS methods—namely, Condensed Nearest Neighbor, Local Set-based Smoother, and Local Set Border Selector—can reduce the size of the training set without effectiveness losses. Furthermore, in the case of fine-tuning the Transformer methods, the IS methods reduce the amount of data needed, without losing effectiveness and with considerable training-time gains.


1 November 2021 in Information Systems
Verified
Journal Article
Closed access
HWHeitor Werneck
Heitor Werneck
NSNícollas Silva
Nícollas Silva
MVMatheus Viana
Matheus Viana
APAdriano C.M. Pereira
Adriano C.M. Pereira
FMFernando Mourão
Fernando Mourão
LRLeonardo Rocha
Leonardo Rocha

17 January 2025 in ACM Transactions on Information Systems
Verified
Journal Article
Open access
WCWashington Cunha
Washington Cunha
AMAlejandro Moreo
Alejandro Moreo
AEAndrea Esuli
Andrea Esuli
FSFabrizio Sebastiani
Fabrizio Sebastiani
LRLeonardo Rocha
Leonardo Rocha
MGMarcos André Gonçalves
Marcos André Gonçalves
Fine-tuning transformer-based deep-learning models are currently at the forefront of natural language processing (NLP) and information retrieval (IR) tasks. However, fine-tuning these transformers for specific tasks, especially when dealing with ever-expanding volumes of data, constant retraining requirements, and budget constraints, can be computationally and financially costly, requiring substantial energy consumption and contributing to carbon dioxide emissions. This article focuses on advancing the state-of-the-art (SOTA) on instance selection (IS)—a range of document filtering techniques designed to select the most representative documents for the sake of training. The objective is to either maintain or enhance classification effectiveness while reducing the overall training (fine-tuning) total processing time. In our prior research, we introduced the E2SC framework, a redundancy-oriented IS method focused on transformers and large datasets—currently the state-of-the-art in IS. Nonetheless, important research questions remained unanswered in our previous work, mostly due to E2SC’s sole emphasis on redundancy. In this article, we take our research a step further by proposing biO-IS— an extended bi - o bjective i nstance s election solution, a novel IS framework aimed at simultaneously removing redundant and noisy instances from the training. biO-IS estimates redundancy based on scalable, fast, and calibrated weak classifiers and captures noise with the support of a new entropy-based step. We also propose a novel iterative process to estimate near-optimum reduction rates for both steps. Our extended solution is able to reduce the training sets by 41% on average (up to 60%) while maintaining the effectiveness in all tested datasets, with speedup gains of 1.67 on average (up to 2.46x). No other baseline, not even our previous SOTA solution, was capable of achieving results with this level of quality, considering the tradeoff among training reduction, effectiveness, and speedup. To ensure reproducibility, our documentation, code, and datasets can be accessed on GitHub— https://github.com/waashk/bio-is .