Balázs Szalontai

Mr. Balázs Szalontai

orcid

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2023 in Infocommunications journal
Verified
Journal Article
Closed access
BSBalázs Szalontai
Balázs Szalontai
PBPéter Bereczky
Péter Bereczky
DHDániel Horpácsi
Dániel Horpácsi

4 August 2021
Verified
Book Chapter
Closed access
PDF file
BSBalázs Szalontai
Balázs Szalontai
AVAndrás Vadász
András Vadász
ZBZsolt Richárd Borsi
Zsolt Richárd Borsi
TVTeréz A. Várkonyi
Teréz A. Várkonyi
BPBalázs Pintér
Balázs Pintér
TGTibor Gregorics
Tibor Gregorics

14 July 2025 in Software
Verified
Journal Article
Open access
PDF file
BSBalázs Szalontai
Balázs Szalontai
BMBalázs Márton
Balázs Márton
BPBalázs Pintér
Balázs Pintér
TGTibor Gregorics
Tibor Gregorics
Benchmark results for large language models often show inconsistencies across different studies. This paper investigates the challenges of reproducing these results in automatic bugfixing using LLMs, on the HumanEvalFix benchmark. To determine the cause of the differing results in the literature, we attempted to reproduce a subset of them by evaluating 12 models in the DeepSeekCoder, CodeGemma, CodeLlama, and WizardCoder model families, in different sizes and tunings. A total of 35 unique results were reported for these models across studies, of which we successfully reproduced 12. We identified several relevant factors that influenced the results. The base models can be confused with their instruction-tuned variants, making their results better than expected. Incorrect prompt templates or generation length can decrease benchmark performance, as well as using 4-bit quantization. Using sampling instead of greedy decoding can increase the variance, especially with higher temperature values. We found that precision and 8-bit quantization have less influence on benchmark results.

20 August 2023
Verified
Book Chapter
Closed access
PDF file
BSBalázs Szalontai
Balázs Szalontai
ÁKÁkos Kukucska
Ákos Kukucska
AVAndrás Vadász
András Vadász
BPBalázs Pintér
Balázs Pintér
TGTibor Gregorics
Tibor Gregorics

23 October 2024
Verified
Book Chapter
Closed access
PDF file
BSBalázs Szalontai
Balázs Szalontai
AVAndrás Vadász
András Vadász
TMTamás Márton
Tamás Márton
BPBalázs Pintér
Balázs Pintér
TGTibor Gregorics
Tibor Gregorics

29 April 2026 in Software
Verified
Journal Article
Open access
PDF file
TMTamás Márton
Tamás Márton
BSBalázs Szalontai
Balázs Szalontai
BPBalázs Pintér
Balázs Pintér
TGTibor Gregorics
Tibor Gregorics
Refactoring is essential for developing maintainable software. Using Large Language Models in software engineering is widespread, but compared to well-established domains such as code generation, reliable refactoring is still relatively underexplored. In this paper, we perform a broad analysis on the refactoring capabilities of small open-weight language models (SLMs) by evaluating 12 models on 3453 Python programs. Our study focuses on the two defining aspects of refactoring: behavior preservation and code quality improvement. We evaluate these properties using unit tests and various code metrics. Across models ranging from 0.5B to 8 B parameters, most models improve code quality. Larger models are more reliable, as they preserve behavior more consistently. Reasoning models often make more significant changes while refactoring. Allowing models to generate reasoning traces improves performance, but only for models larger than 4B. For smaller models, reasoning in fact reduces refactoring reliability. The difficulty of the underlying task affects refactoring performance, with more complex tasks associated with higher failure rates. Our results indicate that current open SLMs can support refactoring tasks, especially larger ones with reasoning capabilities, but they are best used with human oversight.