AI‐based chatbots are increasingly used to automate clinical documentation, but their efficacy in generating specialized nutritional prescriptions for hospital discharge remains underexplored. This preliminary study evaluated the performance of a prominent AI chatbot in producing clinically valid nutritional guidelines. A specialist committee of registered dietitians selected 16 common medical and surgical pathologies. Standardized prompts were used to generate nutritional discharge guidelines from the chatbot. The same committee then evaluated the AI‐generated texts for technical accuracy and content presentation on a 0–10 scale (approval score ≥ 7.0). Inter‐rater reliability was assessed using the intraclass correlation coefficient (ICC) and Cohen’s Kappa. Overall, 50% (8/16) of the AI‐generated prescriptions met the predefined approval threshold. Performance was higher for medical pathologies (mean score: 7.1 ± 1.2) compared to surgical pathologies (6.6 ± 1.4), although this difference was not statistically significant ( p > 0.05). Inter‐rater reliability was substantial (ICC > 0.72; Kappa > 0.62). The findings indicate that AI chatbots hold promise as supplementary tools for drafting nutritional discharge summaries, potentially reducing administrative workload. However, their variable performance underscores the indispensable need for rigorous review and validation by qualified healthcare professionals before any clinical application.
Purpose: To evaluate the accuracy of ChatGPT carbohydrate estimates for commonly consumed Brazilian foods with standardised composition, and to compare free-tier with subscription-tier versions and simplified with elaborated prompts. Methods: A 2 × 2 factorial design covered the 20 most frequently acquired foods in the Brazilian Household Budget Survey (POF 2017–2018). Each food was queried once under four conditions: model tier (GPT-5.1 Instant, free; GPT-5.1 Thinking, subscription) × prompt complexity (lay user; expert). Following the ISO 15197 architecture, an estimate was clinically accurate when its deviation from the Brazilian Diabetes Society (SBD) reference was within 1.0 g per 100 g or 15%, whichever was more permissive. Analyses accounted for the clustering of four observations within each food (Bland–Altman for clustered data, within-food paired tests, linear mixed models with a food random intercept). Results: ChatGPT returned usable estimates for all 80 queries; overall clinical accuracy was 85.0% (68/80; 95% CI 75.3–92.0). No evidence of differences was observed between model tiers or prompt strategies (85.0% in all four conditions; exact McNemar, p = 1.000). Mean absolute error ranged from 1.14 to 1.43 g per 100 g. Agreement analysis showed a small, non-significant tendency towards underestimation (− 0.37 g per 100 g; 95% limits of agreement − 4.42 to + 3.69; p = 0.389). Conclusion: Under controlled conditions, 85.0% of estimates fell within this accuracy margin, and no evidence was found that model tier or prompt complexity influenced estimation error. These findings do not establish clinical safety, effectiveness, equivalence between tiers, or suitability for unsupervised patient use.