Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.
Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.
18 March 2010 in American Journal of Neuroradiology
BACKGROUND AND PURPOSE: Cranial nerve palsies are regularly observed in patients with arteriovenous fistulas of the cavernous sinus. The purpose of our study was to determine the long-term clinical outcome—with a special focus on extra-ocular muscular dysfunctions—in patients who had undergone endovascular treatment of a cavernous sinus fistula with detachable coils. MATERIALS AND METHODS: Sixteen patients were recalled for an ophthalmoneurologic control examination (mean interval of 4.4 years). The mRS and the EQ-5D questionnaire were used for the description of general outcome. Age, duration of symptoms, character of the fistula (direct, dural), and coil volume were tested to assess their relevance for persistent symptoms. RESULTS: All patients displayed complete regression of chemosis, exophthalmus, and pulsating tinnitus with no evidence of recurrences. Oculomotor disturbances persisted in 9 of 13 patients and caused permanent diplopia in 7 patients. In 15 patients a mRS score of 1 or 2 was achieved; however, 7 patients reported some limitations in life quality (EQ-5D). A significant correlation was found between coil volume and persistent diplopia (P = .032) and persistent cranial nerve VI paresis (P = .037). CONCLUSIONS: Coil embolization of the cavernous sinus led to durable closure of AVF and reliable regression of acute symptoms. However, long-term follow-up showed a 44% rate of persistent cranial nerve deficits with disturbances of oculomotor and visual functions. This may be explained by the underlying fistula size itself and/or the space-occupying effect of the coils. As neuro-ophthalmologic outcome is crucial for control of therapeutic success, patients should be routinely examined by ophthalmologists.
10 January 2024 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.