Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing.
31 August 2017 in American Journal of Neuroradiology
INTRODUCTION: Functional MR imaging is increasingly being used for presurgical language assessment in the treatment of patients with brain tumors, epilepsy, vascular malformations, and other conditions. The inherent complexity of fMRI, which includes numerous processing steps and selective analyses, is compounded by institution-unique approaches to patient training, paradigm choice, and an eclectic array of postprocessing options from various vendors. Consequently, institutions perform fMRI in such markedly different manners that data sharing, comparison, and generalization of results are difficult. The American Society of Functional Neuroradiology proposes widespread adoption of common fMRI language paradigms as the first step in countering this lost opportunity to advance our knowledge and improve patient care. LANGUAGE PARADIGM REVIEW PROCESS: A taskforce of American Society of Functional Neuroradiology members from multiple institutions used a broad literature review, member polls, and expert opinion to converge on 2 sets of standard language paradigms that strike a balance between ease of application and clinical usefulness. ASFNR RECOMMENDATIONS: The taskforce generated an adult language paradigm algorithm for presurgical language assessment including the following tasks: Sentence Completion, Silent Word Generation, Rhyming, Object Naming, and/or Passive Story Listening. The pediatric algorithm includes the following tasks: Sentence Completion, Rhyming, Antonym Generation, or Passive Story Listening. DISCUSSION: Convergence of fMRI language paradigms across institutions offers the first step in providing a “Rosetta Stone” that provides a common reference point with which to compare and contrast the usefulness and reliability of fMRI data. From this common language task battery, future refinements and improvements are anticipated, particularly as objective measures of reliability become available. Some commonality of practice is a necessary first step to develop a foundation on which to improve the clinical utility of this field.