Large language models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM, an extension of the TRIPOD + artificial intelligence statement, addressing the unique challenges of LLMs in biomedical applications. TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories. Developed through an expedited Delphi process and expert consensus, TRIPOD-LLM emphasizes transparency, human oversight and task-specific performance reporting. We also introduce an interactive website (https://tripod-llm.vercel.app/) facilitating easy guideline completion and PDF generation for submission. As a living document, TRIPOD-LLM will evolve with the field, aiming to enhance the quality, reproducibility and clinical applicability of LLM research in healthcare through comprehensive reporting.

Introduction: The Brazilian Multilabel Ophthalmological Dataset (BRSET) addresses the scarcity of publicly available ophthalmological datasets in Latin America. BRSET comprises 16,266 color fundus retinal photos from 8,524 Brazilian patients, aiming to enhance data representativeness, serving as a research and teaching tool. It contains sociodemographic information, enabling investigations into differential model performance across demographic groups. Methods: Data from three São Paulo outpatient centers yielded demographic and medical information from electronic records, including nationality, age, sex, clinical history, insulin use, and duration of diabetes diagnosis. A retinal specialist labeled images for anatomical features (optic disc, blood vessels, macula), quality control (focus, illumination, image field, artifacts), and pathologies (e.g., diabetic retinopathy). Diabetic retinopathy was graded using International Clinic Diabetic Retinopathy and Scottish Diabetic Retinopathy Grading. Validation used a ConvNext model trained during 50 epochs using a weighted cross entropy loss to avoid overfitting, with 70% training (20% validation), and 30% testing subsets. Performance metrics included area under the receiver operating curve (AUC) and Macro F1-score. Saliency maps were calculated for interpretability. Results: BRSET comprises 65.1% Canon CR2 and 34.9% Nikon NF5050 images. 61.8% of the patients are female, and the average age is 57.6 (± 18.26) years. Diabetic retinopathy affected 15.8% of patients, across a spectrum of disease severity. Anatomically, 20.2% showed abnormal optic discs, 4.9% abnormal blood vessels, and 28.8% abnormal macula. A ConvNext V2 model was trained and evaluated BRSET in four prediction tasks: “binary diabetic retinopathy diagnosis (Normal vs Diabetic Retinopathy)” (AUC: 97, F1: 89); “3 class diabetic retinopathy diagnosis (Normal, Proliferative, Non-Proliferative)” (AUC: 97, F1: 82); “diabetes diagnosis” (AUC: 91, F1: 83); “sex classification” (AUC: 87, F1: 70). Discussion: BRSET is the first multilabel ophthalmological dataset in Brazil and Latin America. It provides an opportunity for investigating model biases by evaluating performance across demographic groups. The model performance of three prediction tasks demonstrates the value of the dataset for external validation and for teaching medical computer vision to learners in Latin America using locally relevant data sources.

Background: Various surrogate markers of insulin resistance have been developed, capable of predicting coronary artery disease (CAD) without the need to detect serum insulin. For accurate prediction, they depend only on glucose and lipid profiles, as well as anthropometric features. However, there is still no agreement on the most suitable one for predicting CAD. Methods: We followed a cohort of 2,000 individuals, ranging in age from 20 to 74, for a duration of 9.9 years. We utilized multivariate Cox proportional hazard models to investigate the association between TyG-index, TyG-BMI, TyG-WC, TG/HDL, plus METS-IR and the occurrence of CAD. The receiver operating curve (ROC) was employed to compare the predictive efficacy of these indices and their corresponding cutoff values for predicting CAD. We also used three distinct embedded feature selection methods: LASSO, Random Forest feature selection, and the Boruta algorithm, to evaluate and compare surrogate markers of insulin resistance in predicting CAD. In addition, we utilized the ceteris paribus profile on the Random Forest model to illustrate how the model’s predictive performance is affected by variations in individual surrogate markers, while keeping all other factors consistent in a diagram. Results: The TyG-index was the only surrogate marker of insulin resistance that demonstrated an association with CAD in fully adjusted model (HR: 2.54, CI: 1.34–4.81). The association was more prominent in females. Moreover, it demonstrated the highest area under the ROC curve (0.67 [0.63–0.7]) in comparison to other surrogate indices for insulin resistance. All feature selection approaches concur that the TyG-index is the most reliable surrogate insulin resistance marker for predicting CAD. Based on the Ceteris paribus profile of Random Forest the predictive ability of the TyG-index increased steadily after 9 with a positive slope, without any decline or leveling off. Conclusion: Due to the simplicity of assessing the TyG-index with routine biochemical assays and given that the TyG-index was the most effective surrogate insulin resistance index for predicting CAD based on our results, it seems suitable for inclusion in future CAD prevention strategies.

Background: An accurate prognostic tool is essential to aid clinical decision-making (e.g., patient triage) and to advance personalized medicine. However, such a prognostic tool is lacking for acute pancreatitis (AP). Increasingly machine learning (ML) techniques are being used to develop high-performing prognostic models in AP. However, methodologic and reporting quality has received little attention. High-quality reporting and study methodology are critical for model validity, reproducibility, and clinical implementation. In collaboration with content experts in ML methodology, we performed a systematic review critically appraising the quality of methodology and reporting of recently published ML AP prognostic models. Methods/findings: Using a validated search strategy, we identified ML AP studies from the databases MEDLINE and EMBASE published between January 2021 and December 2023. We also searched pre-print servers medRxiv, bioRxiv, and arXiv for pre-prints registered between January 2021 and December 2023. Eligibility criteria included all retrospective or prospective studies that developed or validated new or existing ML models in patients with AP that predicted an outcome following an episode of AP. Meta-analysis was considered if there was homogeneity in the study design and in the type of outcome predicted. For risk of bias (ROB) assessment, we used the Prediction Model Risk of Bias Assessment Tool. Quality of reporting was assessed using the Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis—Artificial Intelligence (TRIPOD+AI) statement that defines standards for 27 items that should be reported in publications using ML prognostic models. The search strategy identified 6,480 publications of which 30 met the eligibility criteria. Studies originated from China (22), the United States (4), and other (4). All 30 studies developed a new ML model and none sought to validate an existing ML model, producing a total of 39 new ML models. AP severity (23/39) or mortality (6/39) were the most common outcomes predicted. The mean area under the curve for all models and endpoints was 0.91 (SD 0.08). The ROB was high for at least one domain in all 39 models, particularly for the analysis domain (37/39 models). Steps were not taken to minimize over-optimistic model performance in 27/39 models. Due to heterogeneity in the study design and in how the outcomes were defined and determined, meta-analysis was not performed. Studies reported on only 15/27 items from TRIPOD+AI standards, with only 7/30 justifying sample size and 13/30 assessing data quality. Other reporting deficiencies included omissions regarding human–AI interaction (28/30), handling low-quality or incomplete data in practice (27/30), sharing analytical codes (25/30), study protocols (25/30), and reporting source data (19/30). Conclusions: There are significant deficiencies in the methodology and reporting of recently published ML based prognostic models in AP patients. These undermine the validity, reproducibility, and implementation of these prognostic models despite their promise of superior predictive accuracy. Registration: Research Registry (reviewregistry1727)

This paper introduces mBRSET, the first publicly available diabetic retinopathy retina dataset captured using handheld retinal cameras in real-life, high-burden scenarios, comprising 5,164 images from 1,291 patients of diverse backgrounds. This dataset addresses the lack of ophthalmological data in low- and middle-income countries (LMICs) by providing a cost-effective and accessible solution for ocular screening and management. Portable retinal cameras enable applications outside traditional hospital settings, such as community health screenings and telemedicine consultations, thereby democratizing healthcare. Extensive metadata that are typically unavailable in other datasets, including age, sex, diabetes duration, treatments, and comorbidities, are also recorded. To validate the utility of mBRSET, state-of-the-art deep models, including ConvNeXt V2, Dino V2, and SwinV2, were trained for benchmarking, achieving high accuracy in clinical tasks diagnosing diabetic retinopathy, and macular edema; and in fairness tasks predicting education and insurance status. The mBRSET dataset serves as a resource for developing AI algorithms and investigating real-world applications, enhancing ophthalmological care in resource-constrained environments.

Artificial intelligence (AI) algorithms have the potential to revolutionize healthcare, but their successful translation into clinical practice has been limited. One crucial factor is the data used to train these algorithms, which must be representative of the population. However, most healthcare databases are derived from high-income countries, leading to non-representative models and potentially exacerbating health inequities. This review focuses on the landscape of health-related open datasets in Latin America, aiming to identify existing datasets, examine data-sharing frameworks, techniques, platforms, and formats, and identify best practices in Latin America. The review found 61 datasets from 23 countries, with the DATASUS dataset from Brazil contributing to the majority of articles. The analysis revealed a dearth of datasets created by the authors themselves, indicating a reliance on existing open datasets. The findings underscore the importance of promoting open data in Latin America. We provide recommendations for enhancing data sharing in the region.