Luc Builtjes

Luc Builtjes

Auto-generated profile, based on publicly available information

Luc Builtjes

Diagnostic Image Analysis Group, Medical Imaging Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, Nijmegen, 6525 GA, The Netherlands, 31 614021323

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 September 2025 in JAMIA Open
Verified
Journal Article
Open access
LBLuc Builtjes
Luc Builtjes
JBJoeran Bosma
Joeran Bosma
MPMathias Prokop
Mathias Prokop
BGBram van Ginneken
Bram van Ginneken
AHAlessa Hering
Alessa Hering
Objective: We aimed to evaluate the zero-shot performance of open-source generative large language models (LLMs) on clinical information extraction from Dutch medical reports using the Diagnostic Report Analysis: General Optimization of NLP (DRAGON) benchmark. Methods: We developed and released the llm_extractinator framework, a scalable, open-source tool for automating information extraction from clinical texts using LLMs. We evaluated 9 multilingual open-source LLMs across 28 tasks in the DRAGON benchmark, covering classification, regression, and named entity recognition (NER). All tasks were performed in a zero-shot setting. Model performance was quantified using task-specific metrics and aggregated into a DRAGON utility score. Additionally, we investigated the effect of in-context translation to English. Results: Llama-3.3-70B achieved the highest utility score (0.760), followed by Phi-4-14B (0.751), Qwen-2.5-14B (0.748), and DeepSeek-R1-14B (0.744). These models outperformed or matched a fine-tuned RoBERTa baseline on 17 of 28 tasks, particularly in regression and structured classification. NER performance was consistently low across all models. Translation to English consistently reduced performance. Discussion: Generative LLMs demonstrated strong zero-shot capabilities on clinical natural language processing tasks involving structured inference. Models around 14B parameters performed well overall, with Llama-3.3-70B leading but at high computational cost. Generative models excelled in regression tasks, but were hindered by token-level output formats for NER. Translation to English reduced performance, emphasizing the need for native language support. Conclusion: Open-source generative LLMs provide a viable zero-shot alternative for clinical information extraction from Dutch medical texts, particularly in low-resource and multilingual settings.

30 April 2026 in JMIR Medical Informatics
Verified
Journal Article
Open access
ASAnoeska Schipper
Anoeska Schipper
PBPeter Belgers
Peter Belgers
RORory David O'COnnor
Rory David O'COnnor
LWLieke van de Wouw
Lieke van de Wouw
LBLuc Builtjes
Luc Builtjes
BGBram van Ginneken
Bram van Ginneken
Background: Most clinically relevant information in emergency department (ED) visits is documented in free text, limiting reuse for research and clinical decision support. Despite growing interest in large language model (LLM)–based feature extraction, very few studies have examined it directly on ED reports. Existing work has mainly addressed binary tasks and rarely evaluated their impact on downstream prediction models. Furthermore, evidence for small multilingual LLMs remains limited, especially for underrepresented languages such as Dutch. Locally deployable LLMs could enable automated feature extraction for decision support systems without increasing physician workload. Objective: We aim to evaluate whether a small open-source LLM (Qwen 2.5:14B) can automatically extract 16 clinical signs and symptoms from ED reports and use these as input for an appendicitis prediction model. LLM performance under minimal and optimized 0-shot prompts was assessed against researcher annotations (reference standard) and physician annotations. Methods: This retrospective study used 336 ED reports from patients presenting with acute abdominal pain to a Dutch teaching hospital (2016-2023). One hundred reports were randomly selected to develop a minimal and an optimized 0-shot prompt strategy. The remaining 236 reports, reserved for evaluation, were annotated by 2 ED physicians and processed by the LLM to extract 16 signs and symptoms, covering binary, multiclass, and multilabel classification tasks. These features were used as input to the HIVE (History, Intake, Vitals, Examination) appendicitis prediction model. LLM extraction accuracy, sensitivity, and specificity were measured against the researcher’s (reference standard) and physician annotations. The HIVE model’s area under the receiver operating characteristic curve was evaluated using LLM-extracted vs physician-annotated features. Results: Among 336 ED reports from patients with acute abdominal pain (median age 41, IQR 22‐62 years, 205/336, 61% female), 50% (167/336) had appendicitis. The LLM achieved weighted average accuracies of 0.910 (95% CI (0.018) with minimal prompts and 0.929 (95% CI ±0.016) with optimized prompts, vs 0.961 (95% CI ±0.012) and 0.951 (95% CI ±0.015) for physicians. Corresponding HIVE model area under the receiver operating characteristic curves were 0.871 (95% CI ±0.019) and 0.911 (95% CI ±0.014) with LLM inputs under the minimal and optimized prompts, compared to 0.917 (95% CI ±0.015) and 0.924 (95% CI ±0.018) for physician inputs. Conclusions: A small locally deployable multilingual LLM can approach physician-level accuracy in extracting structured binary, multiclass, and multilabel clinical data from free-text Dutch ED reports, while preserving patient privacy, interpretability, and statistical transparency for downstream diagnostic modeling.

21 November 2025 in Computers in Biology and Medicine
Verified
Journal Article
Open access
JBJoeran S. Bosma
Joeran S. Bosma
LBLuc Builtjes
Luc Builtjes
ASAnindo Saha
Anindo Saha
JTJasper Twilt
Jasper Twilt
Manolis TsiknakisManolis Tsiknakis
Manolis Tsiknakis
HHHenkjan J. Huisman
Henkjan J. Huisman
Deep learning can mitigate the global radiologist shortage but its development requires large-scale annotated datasets. This study introduces SCALE (Scalable Clinical Annotation with Location Evidence), a fully automated method for generating voxel-level annotations. It uses location priors that are automatically extracted from medical reports, tracked biopsy coordinates, or provided anatomical sectors. We annotated a large-scale dataset comprising 17 896 cases from 16 562 patients across 24 hospitals in 10 countries and 2 continents, using both SCALE and a count-based weakly semisupervised learning (CWSSL) method. An optimized algorithm was developed and trained on these datasets. Evaluation with 1561 cases from 1561 patients across 19 hospitals in 3 countries and 1 continent showed the superiority of the model trained on the dataset with SCALE annotations, achieving a case-level area under the receiver operating characteristic curve of 0.856. This is +0.012 compared to supervised learning (p=0.02), +0.007 compared to training with CWSSL (p=0.12), and +0.006 compared to the Prostate Imaging: Cancer AI (PI-CAI) Ensemble AI System. These results demonstrate that automated, location-guided annotation enables scalable development of AI for clinically significant prostate cancer detection on MRI, surpassing previous methods, and paving the way for broader clinical deployment.

12 March 2026
Verified
Book Chapter
Closed access
PDF file
LBLuc Builtjes
Luc Builtjes
AHAlessa Hering
Alessa Hering

6 October 2026 in SSRN Electronic Journal
Verified
Preprint
Open access
LBLuc Builtjes
Luc Builtjes
JBJoeran S. Bosma
Joeran S. Bosma
JLJudith Lefkes
Judith Lefkes
AVAnouk Veldhuis
Anouk Veldhuis
JGJeroen Geerdink
Jeroen Geerdink
AHAlessa Hering
Alessa Hering
Background and Objective: Clinical free text can only be reused once protected health information (PHI) has been removed or replaced. No openly available transformer model for Dutch de-identification has been trained on real clinical text, and Hiding in Plain Sight (HIPS), which replaces detected PHI with realistic surrogates so that missed identifiers blend in, had no Dutch implementation. We developed GHMSCHRFT, an open pipeline that pairs transformer-based PHI detection with Dutch HIPS, and evaluated both its detection and the residual risk of its output.Methods: An XLM-RoBERTa-large model pretrained on Dutch medical text was fine-tuned on 3,941 reports from two hospitals, annotated for 24 PHI categories, and combined with a Dutch HIPS surrogate module. Detection was tested on 980 internal reports and on 340 reports from a third hospital not used for development, and compared with four existing systems in their default configuration. Detection recall counted a PHI span as found when all its tokens were tagged as PHI, whatever the predicted category, because such spans are replaced. Five readers then searched 100 processed reports, including every internal report with missed PHI, for identifiers that had survived.Results: Entity-level F1-score was 0.943 internally and 0.950 externally, and detection recall was 0.983 and 0.986. On the categories all systems share, GHMSCHRFT reached a detection recall of 0.985 and 0.986, against 0.790 and 0.883 for the strongest comparator, a neural Dutch tagger, and 0.682 or lower for the other three. The 100 reports contained 21 residual PHI spans among 1,039 surrogates. The readers together found 5 of the 21, and no reader's precision exceeded 8.3%, consistent with random selection among all PHI-like candidates.Conclusions: Near-identical performance at an unseen hospital suggests that GHMSCHRFT can be used at other Dutch hospitals without retraining, and it detected substantially more PHI than existing systems. After surrogate replacement, readers could not reliably tell missed identifiers apart from the surrogates around them, although this is a detectability test on 21 spans rather than a re-identification risk assessment. The model, surrogate module, annotation guidelines and evaluation protocol are openly released.

28 July 2025 in arXiv
Verified
Preprint
Open access
PDF file
LBLuc Builtjes
Luc Builtjes
JBJoeran Bosma
Joeran Bosma
MPMathias Prokop
Mathias Prokop
BGBram van Ginneken
Bram van Ginneken
AHAlessa Hering
Alessa Hering
Medical reports contain rich clinical information but are often unstructured and written in domain-specific language, posing challenges for information extraction. While proprietary large language models (LLMs) have shown promise in clinical natural language processing, their lack of transparency and data privacy concerns limit their utility in healthcare. This study therefore evaluates nine open-source generative LLMs on the DRAGON benchmark, which includes 28 clinical information extraction tasks in Dutch. We developed \texttt{llm\_extractinator}, a publicly available framework for information extraction using open-source generative LLMs, and used it to assess model performance in a zero-shot setting. Several 14 billion parameter models, Phi-4-14B, Qwen-2.5-14B, and DeepSeek-R1-14B, achieved competitive results, while the bigger Llama-3.3-70B model achieved slightly higher performance at greater computational cost. Translation to English prior to inference consistently degraded performance, highlighting the need of native-language processing. These findings demonstrate that open-source LLMs, when used with our framework, offer effective, scalable, and privacy-conscious solutions for clinical information extraction in low-resource settings.