Real-world data (RWD), largely stored in unstructured electronic health records (EHRs), are critical for understanding complex diseases like cancer. However, extracting structured information from these narratives is challenging due to linguistic variability, semantic complexity, and privacy concerns. This study evaluates the performance of four locally deployable and small language models (SLMs), LLaMA, Mistral, BioMistral, and MedLLaMA, for information extraction (IE) from Italian EHRs within the APOLLO 11 trial on non-small cell lung cancer (NSCLC). We examined three prompting strategies (zero-shot, few-shot, and annotated few-shot) across English and Italian, involving clinicians with varying expertise to assess prompt design’s impact on accuracy. Results show that general-purpose models (e.g., LLaMA 3.1 8B) outperform biomedical models in most tasks, particularly in extracting binary features. Multiclass variables such as TNM staging, PD-L1, and ECOG were more difficult due to implicit language and lack of standardization. Few-shot prompting and native-language inputs significantly improved performance and reduced hallucinations. Clinical expertise enhanced consistency in annotation, particularly among students using annotated examples. The study confirms that privacy-preserving SLMs can be deployed locally for efficient and secure cancer data extraction. Findings highlight the need for hybrid systems combining SLMs with expert input and underline the importance of aligning clinical documentation practices with SLM capabilities. This is the first study to benchmark SLMs on Italian EHRs and investigate the role of clinical expertise in prompt engineering, offering valuable insights for the future integration of SLMs into real-world clinical workflows.
3 December 2024 in ESMO Real World Data and Digital Oncology
A mapping review of real-world evidence (RWE) publications on targeted therapy (TT) for solid tumours was carried out to describe their characteristics, strengths, and limitations. RWE publications were identified that: (i) focused on TTs in patients with solid tumours; (ii) included study objectives of effectiveness, predictive or prognostic factors, safety or quality of life; (iii) were published between 1 January 2020 and 22 December 2022. Associations between study variables and journal impact factor (IF) were explored through regression and cluster network analyses. Of 7775 publications identified, 1251 were eligible for analysis. The number of publications per year increased over time. Most studies were conducted in Asia (50%), Europe (25%), and North America (17%), with only 8% in more than one country. Data sources were mostly health records (55%) and registries (11%). Most studies were retrospective (85%) and only 16% were population based. Gastrointestinal tumours were the most frequently studied (30%), followed by lung (22%) and breast (21%). The median journal IF was 4.4. Involvement of >10 centres and studies originating from Europe were significantly associated with a higher IF (≥7) in multivariable analysis. Network analysis demonstrated strong associations between countries and the number of publications in specific tumour types. The number of RWE publications on TT for solid tumours is increasing, but studies are heterogeneous, mostly retrospective, and published in low IF journals. International collaboration and promotion of standardised data sources is imperative to enhance the relevance of RWE research to complement clinical guidelines and impact clinical practice.
1 June 2020 in memo - Magazine of European Medical Oncology
Objectives: To describe the scientific evidence regarding adjuvant treatment for stage III and IV cutaneous melanoma and the unresolved issues in this setting. Methods: This review examines the main studies on adjuvant medical therapies approved over the years by the Food and Drug Administration and European Medicines Agency together with the main evidence related to the treatments referred to in the National Comprehensive Cancer Network and European Society of Clinical Oncology guidelines at the time of submission (May 2026) for stage III and IV cutaneous melanoma. A particular focus on immunotherapy (interferon, ipilimumab, and anti-PD-1 antibodies, both as monotherapy and in combination) and targeted therapy with anti-BRAF agents, either as monotherapy or in combination with MEK inhibitors, is given. Besides that, this work also evaluates the role of radiation therapy and addresses some unresolved issues, such as adjuvant therapy in stage IIIA and treatment selection in BRAF-mutated melanoma. Results: Adjuvant therapy for stage III and IV cutaneous melanoma has evolved over the years, starting with interferon and progressing to the use of immunocheckpoint inhibitors and targeted therapy. However, not all treatments that have proven effective in metastatic disease have subsequently played a role in the adjuvant setting. Conclusions: Currently, adjuvant treatment for stage III and IV cutaneous melanoma involves the use of anti-PD-1 antibodies (nivolumab and pembrolizumab) and dabrafenib plus trametinib if the patient has a BRAF V600 mutation. It was not possible to identify the adjuvant therapy of choice for BRAF-mutated melanoma, and several factors must be considered when deciding between immunotherapy and targeted therapy. The role of radiation therapy remains controversial and could be discussed by the multidisciplinary team as part of the adjuvant strategy in selected patients. Likewise, adjuvant therapy for stage IIIA melanoma should be carefully evaluated in light of the risk–benefit ratio.