Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders¹. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications². Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

The heterogeneity of neurodegenerative diseases is a key confound to disease understanding and treatment development, as study cohorts typically include multiple phenotypes on distinct disease trajectories. Here we introduce a machine-learning technique—Subtype and Stage Inference (SuStaIn)—able to uncover data-driven disease phenotypes with distinct temporal progression patterns, from widely available cross-sectional patient studies. Results from imaging studies in two neurodegenerative diseases reveal subgroups and their distinct trajectories of regional neurodegeneration. In genetic frontotemporal dementia, SuStaIn identifies genotypes from imaging alone, validating its ability to identify subtypes; further the technique reveals within-genotype heterogeneity. In Alzheimer’s disease, SuStaIn uncovers three subtypes, uniquely characterising their temporal complexity. SuStaIn provides fine-grained patient stratification, which substantially enhances the ability to predict conversion between diagnostic categories over standard models that ignore subtype (p = 7.18 × 10⁻⁴) or temporal stage (p = 3.96 × 10⁻⁵). SuStaIn offers new promise for enabling disease subtype discovery and precision medicine.

Objective: Gray matter (GM) atrophy occurs in all multiple sclerosis (MS) phenotypes. We investigated whether there is a spatiotemporal pattern of GM atrophy that is associated with faster disability accumulation in MS.Methods: We analyzed 3,604 brain high‐resolution T1‐weighted magnetic resonance imaging scans from 1,417 participants: 1,214 MS patients (253 clinically isolated syndrome [CIS], 708 relapsing‐remitting [RRMS], 128 secondary‐progressive [SPMS], and 125 primary‐progressive [PPMS]), over an average follow‐up of 2.41 years (standard deviation [SD] = 1.97), and 203 healthy controls (HCs; average follow‐up = 1.83 year; SD = 1.77), attending seven European centers. Disability was assessed with the Expanded Disability Status Scale (EDSS). We obtained volumes of the deep GM (DGM), temporal, frontal, parietal, occipital and cerebellar GM, brainstem, and cerebral white matter. Hierarchical mixed models assessed annual percentage rate of regional tissue loss and identified regional volumes associated with time‐to‐EDSS progression.Results: SPMS showed the lowest baseline volumes of cortical GM and DGM. Of all baseline regional volumes, only that of the DGM predicted time‐to‐EDSS progression (hazard ratio = 0.73; 95% confidence interval, 0.65, 0.82; p < 0.001): for every standard deviation decrease in baseline DGM volume, the risk of presenting a shorter time to EDSS worsening during follow‐up increased by 27%. Of all longitudinal measures, DGM showed the fastest annual rate of atrophy, which was faster in SPMS (–1.45%), PPMS (–1.66%), and RRMS (–1.34%) than CIS (–0.88%) and HCs (–0.94%; p < 0.01). The rate of temporal GM atrophy in SPMS (–1.21%) was significantly faster than RRMS (–0.76%), CIS (–0.75%), and HCs (–0.51%). Similarly, the rate of parietal GM atrophy in SPMS (–1.24‐%) was faster than CIS (–0.63%) and HCs (–0.23%; all p values <0.05). Only the atrophy rate in DGM in patients was significantly associated with disability accumulation (beta = 0.04; p < 0.001).Interpretation: This large, multicenter and longitudinal study shows that DGM volume loss drives disability accumulation in MS, and that temporal cortical GM shows accelerated atrophy in SPMS than RRMS. The difference in regional GM atrophy development between phenotypes needs to be taken into account when evaluating treatment effect of therapeutic interventions. Ann Neurol 2018;83:210–222