Clinical documentation, particularly the hospital discharge report (HDR), is essential for ensuring continuity of care, yet its preparation is time-consuming and places a considerable clinical and administrative burden on healthcare professionals. Recent advancements in Generative Artificial Intelligence (GenAI) and the use of prompt engineering in large language models (LLMs) offer opportunities to automate parts of this process, improving efficiency and documentation quality while reducing administrative workload. This study aims to design a digital system based on LLMs capable of automatically generating HDRs using information from clinical course notes and emergency care reports. The system was developed through iterative cycles, integrating various instruction flows and evaluating five different LLMs combined with prompt engineering strategies and agent-based architectures. Throughout the development, more than 60 discharge reports were generated and assessed, leading to continuous system refinement. In the production phase, 40 pneumology discharge reports were produced, receiving positive feedback from physicians, with an average score of 2.9 out of 4, indicating the system’s usefulness, with only minor edits needed in most cases. The ongoing expansion of the system to additional services and its integration within a hospital electronic system highlights the potential of LLMs, when combined with effective prompt engineering and agent-based architectures, to generate high-quality medical content and provide meaningful support to healthcare professionals. Hospital discharge reports (HDRs) are pivotal for continuity of care but consume substantial clinician time. Generative AI systems based on large language models (LLMs) could streamline this process, provided they deliver accurate, multilingual, and workflow-compatible outputs. We pursued a three-stage, design-science approach. Proof-of-concept: five state-of-the-art LLMs were benchmarked with multi-agent prompting to produce sample HDRs and define the optimal agent structure. Prototype: 60 HDRs spanning six specialties were generated and compared with clinician originals using ROUGE with average scores compatible with specialized news summarizing models in Spanish and Catalan (lower scores). A qualitative audit of 27 HDR pairs showed recurrent divergences in medication dose (56%) and social context (52%). Pilot deployment: The AI-HDR service was embedded in the hospital’s electronic health record. In the pilot, 47 HDRs were autogenerated in real-world settings and reviewed by attending physicians. Missing information and factual errors were flagged in 53% and 47% of drafts, respectively, while written assessments diminished the importance of these errors. An LLM-driven, agent-orchestrated pipeline can safely draft real-world HDRs, cutting administrative overhead while achieving clinician-acceptable quality, not without errors that require human supervision. Future work should refine specialty-specific prompts to curb omissions, add temporal consistency checks to prevent outdated data propagation, and validate time savings and clinical impact in multi-center trials.

Background/Objectives: The hospital discharge report is a critical document for care continuity that generates a substantial administrative burden for clinicians. Generative artificial intelligence (AI) offers the potential to reduce this burden while improving documentary quality. This study aims to compare, under real-world conditions with a GDPR-oriented architecture based on prior local anonymisation, the quality of AI-assisted discharge reports produced by the IAIA tool against those drafted by the responsible physician (INF). Methods: A retrospective, paired, expert-evaluation study was conducted at a Spanish university hospital. One hundred and twenty consecutive clinical cases from nine departments were included (240 reports total). Each case was independently evaluated by one of ten primary care physicians using a structured rubric covering 13 clinical dimensions (ordinal scale 1–3) and a global rating scale (1–10). The Wilcoxon signed-rank test was applied to all paired comparisons; effect size was estimated using the paired rank-biserial correlation (r). Results: IAIA achieved a significantly higher overall mean rating than INF (8.14 vs. 7.30 out of 10; p < 0.0001; r ≈ 0.76, large effect). IAIA was superior in 9 of 13 clinical dimensions, with the largest gains in family history, principal diagnosis hierarchy, and structured listing of secondary diagnoses. INF retained an advantage only in allergies and intolerances (2.69 vs. 2.46; p = 0.002), where IAIA tended to use generic formulas. Three dimensions showed no significant difference (prior treatment, physical examination, procedures). Conclusions: AI-assisted discharge reports received higher expert-rated documentary quality scores in a non-blinded paired evaluation across most evaluated dimensions. The physician-written report retained an advantage only in the safety-critical allergy domain, where allergy information must not be inferred by the model but sourced from verified structured fields or explicitly flagged as pending physician validation, supporting the need for a supervised hybrid model in which AI generates the initial draft while the clinician mandatorily validates sensitive content. Prior local anonymisation constitutes a GDPR-oriented approach to generative AI deployment in European hospital settings, substantially reducing the risk of disclosure of identifiable clinical information.

Surgical site infection (SSI) surveillance is typically performed by infection control teams through manual review. This approach is resource-intensive and is prone to subjectivity. This study evaluates the performance of a large language model as a screening tool to reduce chart-review workload by flagging records likely to contain SSI evidence in clinical notes within the electronic health record (EHR) after colorectal surgery. A retrospective diagnostic accuracy study was performed in elective colorectal surgery procedures (2010-2023). Narrative EHRs from the operative admission and early post-discharge period were automatically extracted, anonymized, and analyzed with a standardized prompt. ChatGPT-4o outputs (infected vs non-infected; SSI level) were compared with the results of manual surveillance based on full-chart review. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were calculated from two-by-two contingency tables (95% CIs). A total of 1157 cases were analyzed. The reference standard identified an overall SSI prevalence of 11.32% (superficial incisional 3.28%, deep incisional 1.30%, organ/space 6.74%). For overall SSI detection, ChatGPT-4o-based classification achieved a sensitivity of 86.3% and a specificity of 85.7%, with PPV 43.5% and NPV 98%. Overall accuracy was 85.7%; (95% CI: 83.6-87.6), and agreement was substantial (PABAK 0.72). For superficial and deep incisional SSI, sensitivities were low (21.1% and 20%) despite high specificities (98.7% and 99.7%), whereas performance for organ/space SSI was similar to overall SSI (sensitivity 85.9%, specificity 84.6%). Under routine-prevalence conditions, ChatGPT-4o-based analysis of narrative EHR demonstrated good detection of overall SSI and organ/space-SSI, and a high ability to rule out SSI. These findings support its use as a screening layer within semi-automated surveillance workflows. Reliable detection of incisional SSI remains limited and may require improved documentation signals and/or integration of structured data sources. gov: NCT06626399.

Manual surveillance of surgical site infections (SSIs) after colorectal surgery is resource-intensive, limiting scalability. Semiautomated algorithms based on structured electronic health record (EHR) data may maintain high case-finding sensitivity while reducing workload. A retrospective diagnostic-accuracy study was conducted in a teaching hospital participating in a nationwide SSI surveillance programme. All elective colorectal procedures performed between January 2010 and December 2023 were included. SSIs were classified according to CDC-NHSN/ECDC criteria. Eight binary EHR-derived "alerts" were combined into a composite rule (any alert positive). Manual surveillance served as the reference standard. Performance was assessed overall, by SSI depth (superficial, deep, organ/space), and by procedure type (colon vs rectal). Discrimination (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated with 95 % confidence intervals (CIs). A total of 1213 patients (1085 colon; 128 rectal) were included. The overall SSI incidence was 11.2 % (3.1 % superficial, 1.2 % deep, 6.8 % organ/space). The composite alert achieved an AUC of 0.859 (95 % CI 0.838-0.878) for any SSI, with sensitivity 0.721, specificity 0.876, PPV 0.424, and NPV 0.961. At this operating point, 19 % of procedures would be flagged for manual verification, corresponding to an estimated 81 % reduction in full chart reviews. Discrimination was highest for organ/space infections (AUC 0.919; sensitivity 0.831; specificity 0.911). Performance for deep SSI was intermediate (AUC 0.805), and for superficial SSI, more limited (AUC 0.571). Sensitivity was higher for colon surgery (AUC 0.853) and specificity higher for rectal surgery (AUC 0.881). The structured-data algorithm demonstrated strong overall discrimination and excellent performance for organ/space infections, supporting the feasibility of semiautomated surveillance without compromising detection quality. External and prospective validation, definition of diagnostic safety thresholds, and workload-reduction analyses are required to optimise implementation. Exploration of NLP add-ons may be considered where resources permit. gov: NCT07130656.

Background/Objectives: The hospital discharge report is a critical document for care continuity that generates a substantial administrative burden for clinicians. Generative artificial intelligence (AI) offers the potential to reduce this burden while improving documentary quality. This study aims to compare, under real-world conditions with a GDPR-oriented architecture based on prior local anonymisation, the quality of AI-assisted discharge reports (IAIA) against those drafted by the responsible physician (INF). Methods: A retrospective, paired, expert-evaluation study was conducted at a Spanish university hospital. One hundred and twenty consecutive clinical cases from nine departments were included (240 reports total). Each case was independently evaluated by one of ten primary care physicians using a structured rubric covering 13 clinical dimensions (ordinal scale 1–3) and a global rating scale (1–10). The Wilcoxon signed-rank test was applied to all paired comparisons; effect size was estimated using the paired rank-biserial correlation (r). Results: IAIA achieved a significantly higher overall mean rating than INF (8.14 vs. 7.30 out of 10; p < 0.0001; r ≈ 0.76, large effect). IAIA was nominally superior in 9 of 13 clinical dimensions; after Bonferroni correction for the 13 per-dimension comparisons, six of these differences remained statistically significant, with the largest gains in family history, principal diagnosis hierarchy, and structured listing of secondary diagnoses. INF retained an advantage only in allergies and intolerances (2.69 vs. 2.46; p = 0.002), where IAIA tended to use generic formulas. Three dimensions showed no significant difference (prior treatment, physical examination, procedures). Conclusions: AI-assisted discharge reports received higher expert-rated documentary quality scores in a non-blinded paired evaluation across most evaluated dimensions. The physician-written report retained an advantage only in the safety-critical allergy domain, where allergy information must not be inferred by the model but sourced from verified structured fields or explicitly flagged as pending physician validation, supporting the need for a supervised hybrid model in which AI generates the initial draft while the clinician mandatorily validates sensitive content. Prior local anonymisation constitutes a GDPR-oriented approach to generative AI deployment in European hospital settings, substantially reducing the risk of disclosure of identifiable clinical information.