Roham Koohestani

Roham Koohestani

Auto-generated profile, based on publicly available information

Roham Koohestani

Delft University of Technology, Delft, Netherlands

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 December 2025 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
RKRoham Koohestani
Roham Koohestani
PBPhilippe de Bekker
Philippe de Bekker
BKBegüm Koç
Begüm Koç
MIMaliheh Izadi
Maliheh Izadi
Benchmarks are essential for unified evaluation and reproducibility. The rapid rise of Artificial Intelligence for Software Engineering (AI4SE) has produced numerous benchmarks for tasks such as code generation and bug repair. However, this proliferation has led to major challenges: (1) fragmented knowledge across tasks, (2) difficulty in selecting contextually relevant benchmarks, (3) lack of standardization in benchmark creation, and (4) flaws that limit utility. Addressing these requires a dual approach: systematically mapping existing benchmarks for informed selection and defining unified guidelines for robust, adaptable benchmark development. We conduct a review of 247 studies, identifying 273 AI4SE benchmarks since 2014. We categorize them, analyze limitations, and expose gaps in current practices. Building on these insights, we introduce BenchScout, an extensible semantic search tool for locating suitable benchmarks. BenchScout employs automated clustering with contextual embeddings of benchmark-related studies, followed by dimensionality reduction. In a user study with 22 participants, BenchScout achieved usability, effectiveness, and intuitiveness scores of 4.5, 4.0, and 4.1 out of 5. To improve benchmarking standards, we propose BenchFrame, a unified approach to improve benchmark quality. Applying Bench-Frame to HumanEval yielded HumanEvalNext, which features corrected errors, improved language conversion, higher test coverage, and greater difficulty. Evaluating 10 state-of-the-art code models on HumanEval, HumanEvalPlus, and HumanEvalNext revealed average pass-at-1 drops of 31.22% and 19.94%, respectively, underscoring the need for continuous benchmark refinement. We further examine BenchFrame’s scalability through an agentic pipeline and confirm its generalizability on the MBPP dataset. Lastly, we publicly release the material of our review, user study, and the enhanced benchmark.1

5 July 2026
Verified
Conference Paper
Open access
RKRoham Koohestani
Roham Koohestani
ASAgnia Sergeyuk
Agnia Sergeyuk
DGDavid Gros
David Gros
CSClaudio Spiess
Claudio Spiess
STSergey Titov
Sergey Titov
MIMaliheh Izadi
Maliheh Izadi

12 April 2026
Verified
Conference Paper
Open access
RKRoham Koohestani
Roham Koohestani
PBParham Bateni
Parham Bateni
AEAydin Ebrahimi
Aydin Ebrahimi
BEBehdad Etezadi
Behdad Etezadi
KKKiarash Karimi
Kiarash Karimi
MIMaliheh Izadi
Maliheh Izadi


5 July 2026
Verified
Conference Paper
Open access
RKRoham Koohestani
Roham Koohestani
AAAli Al-Kaswan
Ali Al-Kaswan
JKJonathan Katzy
Jonathan Katzy
MIMaliheh Izadi
Maliheh Izadi
Code Large Language Models are frequently trained on massive datasets containing restrictively licensed source code. This creates urgent data governance and copyright challenges. Membership Inference Attacks (MIAs) can serve as an auditing mechanism to detect unauthorized data usage in models. While attacks like the Loss Attack provide a baseline, more involved methods like Polarized Augment Calibration (PAC) remain underexplored in the code domain. This paper presents an exploratory study evaluating these methods on 3B-7B parameter code models. We find that while PAC generally outperforms the Loss baseline, its effectiveness relies on augmentation strategies that disregard the rigid syntax of code, leading to performance degradation on larger, complex files. To address this, we introduce AST-PAC, a domain-specific adaptation that utilizes Abstract Syntax Tree (AST) based perturbations to generate syntactically valid calibration samples. Preliminary results indicate that AST-PAC improves as syntactic size grows, where PAC degrades, but under-mutates small files and underperforms on alphanumeric-rich code. Overall, the findings motivate future work on syntax-aware and size-adaptive calibration as a prerequisite for reliable provenance auditing of code language models.