With the recent diffusion of access to publicly available large language models (LLMs), common interest in generative artificial-intelligence-based applications for medical purposes has skyrocketed. The increased use of these models by tech-savvy patients for personal health issues calls for a scientific evaluation of whether LLMs provide a satisfactory level of accuracy for treatment decisions. This observational study compares the concordance of treatment recommendations from the popular LLM ChatGPT 3.5 with those of a multidisciplinary tumor board for breast cancer (MTB). The study design builds on previous findings by combining an extended input model with patient profiles reflecting patho- and immunomorphological diversity of primary breast cancer, including primary metastasis and precancerous tumor stages. Overall concordance between the LLM and MTB is reached for half of the patient profiles, including precancerous lesions. In the assessment of invasive breast cancer profiles, the concordance amounts to 58.8%. Nevertheless, as the LLM makes considerably fraudulent decisions at times, we do not identify the current development status of publicly available LLMs to be adequate as a support tool for tumor boards. Gynecological oncologists should familiarize themselves with the capabilities of LLMs in order to understand and utilize their potential while keeping in mind potential risks and limitations.

Purpose: This study investigated the concordance of five different publicly available Large Language Models (LLM) with the recommendations of a multidisciplinary tumor board regarding treatment recommendations for complex breast cancer patient profiles. Methods: Five LLM, including three versions of ChatGPT (version 4 and 3.5, with data access until September 3021 and January 2022), Llama2, and Bard were prompted to produce treatment recommendations for 20 complex breast cancer patient profiles. LLM recommendations were compared to the recommendations of a multidisciplinary tumor board (gold standard), including surgical, endocrine and systemic treatment, radiotherapy, and genetic testing therapy options. Results: GPT4 demonstrated the highest concordance (70.6%) for invasive breast cancer patient profiles, followed by GPT3.5 September 2021 (58.8%), GPT3.5 January 2022 (41.2%), Llama2 (35.3%) and Bard (23.5%). Including precancerous lesions of ductal carcinoma in situ, the identical ranking was reached with lower overall concordance for each LLM (GPT4 60.0%, GPT3.5 September 2021 50.0%, GPT3.5 January 2022 35.0%, Llama2 30.0%, Bard 20.0%). GPT4 achieved full concordance (100%) for radiotherapy. Lowest alignment was reached in recommending genetic testing, demonstrating a varying concordance (55.0% for GPT3.5 January 2022, Llama2 and Bard up to 85.0% for GPT4). Conclusion: This early feasibility study is the first to compare different LLM in breast cancer care with regard to changes in accuracy over time, i.e., with access to more data or through technological upgrades. Methodological advancement, i.e., the optimization of prompting techniques, and technological development, i.e., enabling data input control and secure data processing, are necessary in the preparation of large-scale and multicenter studies to provide evidence on their safe and reliable clinical application. At present, safe and evidenced use of LLM in clinical breast cancer care is not yet feasible.

Purpose: Large language models (LLM) show potential for decision support in breast cancer care. Their use in clinical care is currently prohibited by lack of control over sources used for decision-making, explainability of the decision-making process and health data security issues. Recent development of Small Language Models (SLM) is discussed to address these challenges. This preclinical proof-of-concept study tailors an open-source SLM to the German breast cancer guideline (BC-SLM) to evaluate initial clinical accuracy and technical functionality in a preclinical simulation. Methods: A multidisciplinary tumor board (MTB) is used as the gold-standard to assess the initial clinical accuracy in terms of concordance of the BC-SLM with MTB and comparing it to two publicly available LLM, ChatGPT3.5 and 4. The study includes 20 fictional patient profiles and recommendations for 5 treatment modalities, resulting in 100 binary treatment recommendations (recommended or not recommended). Statistical evaluation includes concordance with MTB in % including Cohen’s Kappa statistic (κ). Technical functionality is assessed qualitatively in terms of local hosting, adherence to the guideline and information retrieval. Results: The overall concordance amounts to 86% for BC-SLM (κ = 0.721, p < 0.001), 90% for ChatGPT4 (κ = 0.820, p < 0.001) and 83% for ChatGPT3.5 (κ = 0.661, p < 0.001). Specific concordance for each treatment modality ranges from 65 to 100% for BC-SLM, 85–100% for ChatGPT4, and 55–95% for ChatGPT3.5. The BC-SLM is locally functional, adheres to the standards of the German breast cancer guideline and provides referenced sections for its decision-making. Conclusion: The tailored BC-SLM shows initial clinical accuracy and technical functionality, with concordance to the MTB that is comparable to publicly-available LLMs like ChatGPT4 and 3.5. This serves as a proof-of-concept for adapting a SLM to an oncological disease and its guideline to address prevailing issues with LLM by ensuring decision transparency, explainability, source control, and data security, which represents a necessary step towards clinical validation and safe use of language models in clinical oncology.

Background: The complex nature of rheumatic diseases poses considerable challenges for clinicians when developing individualized treatment plans. Large language models (LLMs) such as ChatGPT could enable treatment decision support. Objective: To compare treatment plans generated by ChatGPT-3.5 and GPT-4 to those of a clinical rheumatology board (RB). Design/methods: Fictional patient vignettes were created and GPT-3.5, GPT-4, and the RB were queried to provide respective first- and second-line treatment plans with underlying justifications. Four rheumatologists from different centers, blinded to the origin of treatment plans, selected the overall preferred treatment concept and assessed treatment plans’ safety, EULAR guideline adherence, medical adequacy, overall quality, justification of the treatment plans and their completeness as well as patient vignette difficulty using a 5-point Likert scale. Results: 20 fictional vignettes covering various rheumatic diseases and varying difficulty levels were assembled and a total of 160 ratings were assessed. In 68.8% (110/160) of cases, raters preferred the RB’s treatment plans over those generated by GPT-4 (16.3%; 26/160) and GPT-3.5 (15.0%; 24/160). GPT-4’s plans were chosen more frequently for first-line treatments compared to GPT-3.5. No significant safety differences were observed between RB and GPT-4’s first-line treatment plans. Rheumatologists’ plans received significantly higher ratings in guideline adherence, medical appropriateness, completeness and overall quality. Ratings did not correlate with the vignette difficulty. LLM-generated plans were notably longer and more detailed. Conclusion: GPT-4 and GPT-3.5 generated safe, high-quality treatment plans for rheumatic diseases, demonstrating promise in clinical decision support. Future research should investigate detailed standardized prompts and the impact of LLM usage on clinical decisions.

Emerging digital technologies promise to improve breast cancer care, however lack of awareness among clinicians often prevents timely adoption. This study aims to investigate current awareness and intention-to-use of three technologies among breast cancer healthcare professionals (HCP): (1) digital health applications (DHA), (2) artificial intelligence (AI), and (3) blockchain technology (BC). A 22-item questionnaire was designed and administered before and after a 30 min educational presentation highlighting technology implementation examples. Technology awareness and intention-to-use were measured using 7-point Likert scales. Correlations between demographics, technology awareness, intention-to-use, and eHealth literacy (GR-eHEALS scale) were analyzed. 45 HCP completed the questionnaire, of whom 26 (57.8%) were female. Age ranged from 24 to 67 {mean age (SD): 44.93 ± 12.62}. Awareness was highest for DHA (68.9%) followed by AI (66.7%) and BC (24.4%). The presentation led to a non-significant increase of intention-to-use AI {5.37 (±1.81) to 5.83 (±1.64)}. HCPs´ intention-to-use BC after the presentation increased significantly {4.30 (±2.04) to 5.90 (±1.67), p < 0.01}. Mean accumulated score for GR-eHEALS averaged 33.04 (± 6.61). HCPs´ intended use of AI significantly correlated with eHealth literacy (ρ = 0.383; p < 0.01), intention-to-use BC (ρ = 0.591; p < 0.01) and participants´ age (ρ = −0.438; p < 0.01). This study demonstrates the effect that even a short practical presentation can have on HCPs´ intention-to-use emerging digital technologies. Training potential professional users should be addressed alongside the development of new information technologies and is crucial to increase HCPs´ corresponding awareness and intended use.

Acute kidney injury is one of the most frequent and prognostically relevant complications in cardiogenic shock. The purpose of this study was to evaluate the potential effect of the Impella^® pump on hemodynamics and renal organ perfusion in patients with myocardial infarction complicating cardiogenic shock. Between January 2020 and February 2022 patients with infarct-related cardiogenic shock supported with the Impella^® pump were included in this single-center prospective short-term study. Changes in hemodynamics on different levels of Impella^® support were documented with invasive pulmonal arterial catheter. As far as renal function is concerned, renal perfusion was assessed by determining the renal resistive index (RRI) using Doppler sonography. A total of 50 patients were included in the analysis. The increase in the Impella^® output by a mean of 1.0 L/min improved the cardiac index (2.7 ± 0.86 to 3.3 ± 1.1 p < 0.001) and increased central venous oxygen saturation (62.6 ± 11.8% to 67.4 ± 10.5% p < 0.001). On the other side, the systemic vascular resistance (1035 ± 514 N·s/m⁵ to 902 ± 371 N·s/m⁵p = 0.012) and the RRI were significantly reduced (0.736 ± 0.07 to 0.62 ± 0.07 p < 0.001). Furthermore, in the overall cohort, a baseline RRI ≥ 0.8 was associated with a higher frequency of renal replacement therapy (71% vs. 39% p = 0.04), whereas the consequent reduction of the RRI below 0.7 during Impella^® support improved the glomerular filtration rate (GFR) during hospital stay (15 ± 3 days; 53 ± 16 mL/min to 83 ± 16 mL/min p = 0.04). Impella^® support in patients with cardiogenic shock seems to improve hemodynamics and renal organ perfusion. The RRI, a well-known parameter for the early detection of acute kidney injury, can be directly influenced by the Impella^® flow rate. Thus, a targeted control of the RRI by the Impella^® pump could mediate renal organ protection.