Philippe de Bekker

Philippe de Bekker

Auto-generated profile, based on publicly available information

Philippe de Bekker

Electrical Engineering, Mathematics, and Computer Science (EEMCS), Delft University of Technology, Delft, XE, The Netherlands

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 April 2024
Verified
Conference Paper
Open access
TDTim van Dam
Tim van Dam
FHFrank van der Heijden
Frank van der Heijden
PBPhilippe de Bekker
Philippe de Bekker
BNBerend Nieuwschepen
Berend Nieuwschepen
MOMarc Otten
Marc Otten
MIMaliheh Izadi
Maliheh Izadi
Language model-based code completion models have quickly grown in use, helping thousands of developers write code in many different programming languages. However, research on code completion models typically focuses on imperative languages such as Python and JavaScript, which results in a lack of representation for functional programming languages. Consequently, these models often perform poorly on functional languages such as Haskell. To investigate whether this can be alleviated, we evaluate the performance of two language models for code, CodeGPT and UniXcoder, on the functional programming language Haskell. We fine-tune and evaluate the models on Haskell functions sourced from a publicly accessible Haskell dataset on HuggingFace. Additionally, we manually evaluate the models using our novel translated HumanEval dataset. Our automatic evaluation shows that knowledge of imperative programming languages in the pre-training of LLMs may not transfer well to functional languages, but that code completion on functional languages is feasible. Consequently, this shows the need for more high-quality Haskell datasets. A manual evaluation on HumanEval-Haskell indicates CodeGPT frequently generates empty predictions and extra comments, while UniXcoder more often produces incomplete or incorrect predictions. Finally, we release HumanEval-Haskell, along with the fine-tuned models and all code required to reproduce our experiments on GitHub [41].

16 December 2025 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
RKRoham Koohestani
Roham Koohestani
PBPhilippe de Bekker
Philippe de Bekker
BKBegüm Koç
Begüm Koç
MIMaliheh Izadi
Maliheh Izadi
Benchmarks are essential for unified evaluation and reproducibility. The rapid rise of Artificial Intelligence for Software Engineering (AI4SE) has produced numerous benchmarks for tasks such as code generation and bug repair. However, this proliferation has led to major challenges: (1) fragmented knowledge across tasks, (2) difficulty in selecting contextually relevant benchmarks, (3) lack of standardization in benchmark creation, and (4) flaws that limit utility. Addressing these requires a dual approach: systematically mapping existing benchmarks for informed selection and defining unified guidelines for robust, adaptable benchmark development. We conduct a review of 247 studies, identifying 273 AI4SE benchmarks since 2014. We categorize them, analyze limitations, and expose gaps in current practices. Building on these insights, we introduce BenchScout, an extensible semantic search tool for locating suitable benchmarks. BenchScout employs automated clustering with contextual embeddings of benchmark-related studies, followed by dimensionality reduction. In a user study with 22 participants, BenchScout achieved usability, effectiveness, and intuitiveness scores of 4.5, 4.0, and 4.1 out of 5. To improve benchmarking standards, we propose BenchFrame, a unified approach to improve benchmark quality. Applying Bench-Frame to HumanEval yielded HumanEvalNext, which features corrected errors, improved language conversion, higher test coverage, and greater difficulty. Evaluating 10 state-of-the-art code models on HumanEval, HumanEvalPlus, and HumanEvalNext revealed average pass-at-1 drops of 31.22% and 19.94%, respectively, underscoring the need for continuous benchmark refinement. We further examine BenchFrame’s scalability through an agentic pipeline and confirm its generalizability on the MBPP dataset. Lastly, we publicly release the material of our review, user study, and the enhanced benchmark.1

3 March 2023 in Energies
Verified
Journal Article
Open access
PDF file
PBPhilippe de Bekker
Philippe de Bekker
SCSho André Cremers
Sho André Cremers
SNSonam Norbu
Sonam Norbu
David FlynnDavid Flynn
David Flynn
Valentin RobuValentin Robu
Valentin Robu
Given the fundamental role of renewable energy assets in achieving global temperature control targets, new energy management methods are required to efficiently match intermittent renewable generation and demand. Based on analysing various designed cases, this paper explores a number of heuristics for a smart battery scheduling algorithm that efficiently matches available power supply and demand. The core of improvement of the proposed smart battery scheduling algorithm is exploiting future knowledge, which can be realized by current state-of-the-art forecasting techniques, to effectively store and trade energy. The performance of the developed heuristic battery scheduling algorithm using forecast data of demands, generation, and energy prices is compared to a heuristic baseline algorithm, where decisions are made solely on the current state of the battery, demand, and generation. The battery scheduling algorithms are tested using real data from two large-scale smart energy trials in the UK, in addition to various types and levels of simulated uncertainty in forecasts. The results show that when using a battery to store generated energy, on average, the newly proposed algorithm outperforms the baseline algorithm, obtaining up to 20–60% more profit for the prosumer from their energy assets, in cases where the battery is optimally sized and high-quality forecasts are available. Crucially, the proposed algorithm generates greater profit than the baseline method even with large uncertainty on the forecast, showing the robustness of the proposed solution. On average, only 2–12% of profit is lost on generation and demand uncertainty compared to perfect forecasts. Furthermore, the performance of the proposed algorithm increases as the uncertainty decreases, showing great promise for the algorithm as the quality of forecasting keeps improving.