Background: Limited data availability impedes the advancements of artificial intelligence (AI) applications in surgery to date. Federated learning (FL), a novel privacy-focused machine learning technique, introduces a decentralized framework to facilitate multicenter modeling. This study constructs a FL network, including differential privacy (DP), to predict 90-day mortality following colorectal surgery. Methods: Patients undergoing elective colorectal surgery across three tertiary centers (C1–C3) between January 2015 and December 2021 were retrospectively enrolled. Neural networks (NN) for mortality prediction were trained and validated for all three centers individually and after data aggregation using centralized and distributed FL analysis. Local and central DP was then applied as additional data protection framework components. Areas under the receiver operating characteristic (AUROC) and precision-recallcurves (AUPRC) including 95%-confidence intervals were calculated. Results: A total of 2959 patients (mean [SD] age: 56.9 [16.8] years; n = 1677 [56.7%] male) were enrolled. The 90-day mortality rate was 3.1% (n = 92). Local NNs achieved AUROCs of 0.80 ([0.74–0.87]; C1), 0.81 ([0.75–0.87]; C2), and 0.84 ([0.75–0.92]; C3) and corresponding AUPRCs reaching 0.26 [0.11–0.41], 0.31 [0.21–0.41], and 0.21 [0.11–0.32], respectively. An aggregated centralized NN (NN CZ ) achieved an AUROC of 0.81 [0.76–0.85] and AUPRC of 0.29 [0.22–0.36]. Distributed FL (NN FL ) was comparable (AUROC: 0.78 [0.72–0.84], P = 0.67; AUPRC: 0.26 [0.17–0.35], P = 0.44) to the centralized model. Central DP reduced the performance of FL-based prediction by 5% in AUROC (0.74 [0.64–0.84]) and 35% in AUPRC (0.17 [0.10–0.23]). Local DP almost diminished the performance (AUROC: 0.52 [0.49–0.55]; AUPRC: 0.05 [0.03–0.06]). Feature importance analysis revealed age, blood status and the Charlson Comorbidity Index as highest weighted features for both NN CZ and NN FL . Conclusion: Federated learning demonstrates similar performance to centralized machine learning in preoperative mortality prediction, providing an encouraging framework to accelerate the development of future surgical AI applications. Improving data privacy through DP is associated with compromises in model performance.
Background: Machine Learning (ML) is increasingly being adopted in biomedical research, however, its potential for outcome prediction in visceral surgery remains uncertain. This study compares the potential of ML methods for preoperative 90-day mortality (90DM) prediction of an aggregated multi-organ approach to conventional scoring systems and individual organ models. Methods: This retrospective cohort study enrolled patients undergoing major elective visceral surgery between 2014 and 2022 across two tertiary centers. Multiple ML models for preoperative 90DM prediction were trained, externally validated and benchmarked against the American Society of Anesthesiologists (ASA) score and revised Charlson Comorbidity Index (rCCI). Areas under the receiver operating characteristic (AUROC) and precision recall curves (AUPRC) including standard deviations were calculated. Additionally, individual models for esophageal, gastric, intestinal, liver, and pancreatic surgery were developed and compared to an aggregated approach. Results: 7711 cases encompassing 78 features were included. Overall 90DM was 4% (n = 309). An XBoost classifier demonstrated the best performance and high robustness following external validation (AUROC: 0.86 [0.01]; AUPRC: 0.2 [0.04]). All models outperformed the ASA score (AUROC: 0.72; AUPRC: 0.08) and rCCI (AUROC: 0.81; AUPRC: 0.11). rCCI, patient age and C-reactive protein emerged as most decisive model weights. Models for gastric (AUROC: 0.88 [0.13]; AUPRC: 0.24 [0.26]) and intestinal surgery (AUROC: 0.87 [0.05]; AUPRC: 0.17 [0.09]) revealed the highest organ-specific performances, while pancreatic surgery yielded the lowest results (AUROC: 0.66 [0.08]; AUPRC: 0.22 [0.12]). A combined multi-organ approach (AUROC: 0.84 [0.04]; AUPRC: 0.21 [0.06]) demonstrated superiority over the weighted average across all organ-specific models (AUROC: 0.82 [0.07]; AUPRC: 0.2 [0.13]). Conclusion: ML offers robust preoperative risk stratification for 90DM in elective visceral surgery. Leveraging training across multi-organ cohorts may improve accuracy and robustness compared to organ-specific models. Prospective studies are needed to confirm the potential of ML in surgical outcome prediction.
Sensor-based human activity recognition is becoming ever more prevalent. The increasing importance of distinguishing human movements, particularly in healthcare, coincides with the advent of increasingly compact sensors. A complex sequence of individual steps currently characterizes the activity recognition pipeline. It involves separate data collection, preparation, and processing steps, resulting in a heterogeneous and fragmented process. To address these challenges, we present a comprehensive framework, HARE, which seamlessly integrates all necessary steps. HARE offers synchronized data collection and labeling, integrated pose estimation for data anonymization, a multimodal classification approach, and a novel method for determining optimal sensor placement to enhance classification results. Additionally, our framework incorporates real-time activity recognition with on-device model adaptation capabilities. To validate the effectiveness of our framework, we conducted extensive evaluations using diverse datasets, including our own collected dataset focusing on nursing activities. Our results show that HARE’s multimodal and on-device trained model outperforms conventional single-modal and offline variants. Furthermore, our vision-based approach for optimal sensor placement yields comparable results to the trained model. Our work advances the field of sensor-based human activity recognition by introducing a comprehensive framework that streamlines data collection and classification while offering a novel method for determining optimal sensor placement.
Medical applications of machine learning (ML) have experienced a surge in popularity in recent years. The intensive care unit (ICU) is a natural habitat for ML given the abundance of available data from electronic health records. Models have been proposed to address numerous ICU prediction tasks like the early detection of complications. While authors frequently report state-of-the-art performance, it is challenging to verify claims of superiority. Datasets and code are not always published, and cohort definitions, preprocessing pipelines, and training setups are difficult to reproduce. This work introduces Yet Another ICU Benchmark (YAIB), a modular framework that allows researchers to define reproducible and comparable clinical ML experiments; we offer an end-to-end solution from cohort definition to model evaluation. The framework natively supports most open-access ICU datasets (MIMIC III/IV, eICU, HiRID, AUMCdb) and is easily adaptable to future ICU datasets. Combined with a transparent preprocessing pipeline and extensible training code for multiple ML and deep learning models, YAIB enables unified model development. Our benchmark comes with five predefined established prediction tasks (mortality, acute kidney injury, sepsis, kidney function, and length of stay) developed in collaboration with clinicians. Adding further tasks is straightforward by design. Using YAIB, we demonstrate that the choice of dataset, cohort definition, and preprocessing have a major impact on the prediction performance - often more so than model class - indicating an urgent need for YAIB as a holistic benchmarking tool. We provide our work to the clinical ML community to accelerate method development and enable real-world clinical implementations. Software Repository: https://github.com/rvandewater/YAIB.
Risk prediction prior to oncologic esophagectomy is crucial for assisting surgeons and patients in their joint informed decision making. Recently, a new risk prediction model for 90-day mortality after esophagectomy using the International Esodata Study Group (IESG) database was proposed, allowing for the preoperative assignment of patients into different risk categories. However, given the non-linear dependencies between patient- and tumor-related risk factors contributing to cumulative surgical risk, machine learning (ML) may evolve as a novel and more integrated approach for mortality prediction. We evaluated the IESG risk model and compared its performance to ML models. Multiple classifiers were trained and validated on 552 patients from two independent centers undergoing oncologic esophagectomies. The discrimination performance of each model was assessed utilizing the area under the receiver operating characteristics curve (AUROC), the area under the precision–recall curve (AUPRC), and the Matthews correlation coefficient (MCC). The 90-day mortality rate was 5.8%. We found that IESG categorization allowed for adequate group-based risk prediction. However, ML models provided better discrimination performance, reaching superior AUROCs (0.64 [0.63–0.65] vs. 0.44 [0.32–0.56]), AUPRCs (0.25 [0.24–0.27] vs. 0.11 [0.05–0.21]), and MCCs (0.27 ([0.25–0.28] vs. 0.15 [0.03–0.27]). Conclusively, ML shows promising potential to identify patients at risk prior to surgery, surpassing conventional statistics. Still, larger datasets are needed to achieve higher discrimination performances for large-scale clinical implementation in the future.