Echocardiography uses ultrasound technology to capture high temporal and spatial resolution images of the heart and surrounding structures, and is the most common imaging modality in cardiovascular medicine. Using convolutional neural networks on a large new dataset, we show that deep learning applied to echocardiography can identify local cardiac structures, estimate cardiac function, and predict systemic phenotypes that modify cardiovascular risk but not readily identifiable to human interpretation. Our deep learning model, EchoNet, accurately identified the presence of pacemaker leads (AUC = 0.89), enlarged left atrium (AUC = 0.86), left ventricular hypertrophy (AUC = 0.75), left ventricular end systolic and diastolic volumes ( $${R}^{2}$$R2 = 0.74 and $${R}^{2}$$R2 = 0.70), and ejection fraction ( $${R}^{2}$$R2 = 0.50), as well as predicted systemic phenotypes of age ( $${R}^{2}$$R2 = 0.46), sex (AUC = 0.88), weight ( $${R}^{2}$$R2 = 0.56), and height ( $${R}^{2}$$R2 = 0.33). Interpretation analysis validates that EchoNet shows appropriate attention to key cardiac structures when performing human-explainable tasks and highlights hypothesis-generating regions of interest when predicting systemic phenotypes difficult for human interpretation. Machine learning on echocardiography images can streamline repetitive tasks in the clinical workflow, provide preliminary interpretation in areas with insufficient qualified cardiologists, and predict phenotypes challenging for human evaluation.

The development of robust artificial intelligence models for echocardiography has been limited by the availability of annotated clinical data. Here, to address this challenge and improve the performance of cardiac imaging models, we developed EchoCLIP, a vision–language foundation model for echocardiography, that learns the relationship between cardiac ultrasound images and the interpretations of expert cardiologists across a wide range of patients and indications for imaging. After training on 1,032,975 cardiac ultrasound videos and corresponding expert text, EchoCLIP performs well on a diverse range of benchmarks for cardiac image interpretation, despite not having been explicitly trained for individual interpretation tasks. EchoCLIP can assess cardiac function (mean absolute error of 7.1% when predicting left ventricular ejection fraction in an external validation dataset) and identify implanted intracardiac devices (area under the curve (AUC) of 0.84, 0.92 and 0.97 for pacemakers, percutaneous mitral valve repair and artificial aortic valves, respectively). We also developed a long-context variant (EchoCLIP-R) using a custom tokenizer based on common echocardiography concepts. EchoCLIP-R accurately identified unique patients across multiple videos (AUC of 0.86), identified clinical transitions such as heart transplants (AUC of 0.79) and cardiac surgery (AUC 0.77) and enabled robust image-to-text search (mean cross-modal retrieval rank in the top 1% of candidate text reports). These capabilities represent a substantial step toward understanding and applying foundation models in cardiovascular imaging for preliminary interpretation of echocardiographic findings.

Artificial intelligence (AI) has been developed for echocardiography^1–3, although it has not yet been tested with blinding and randomization. Here we designed a blinded, randomized non-inferiority clinical trial (ClinicalTrials.gov ID: NCT05140642; no outside funding) of AI versus sonographer initial assessment of left ventricular ejection fraction (LVEF) to evaluate the impact of AI in the interpretation workflow. The primary end point was the change in the LVEF between initial AI or sonographer assessment and final cardiologist assessment, evaluated by the proportion of studies with substantial change (more than 5% change). From 3,769 echocardiographic studies screened, 274 studies were excluded owing to poor image quality. The proportion of studies substantially changed was 16.8% in the AI group and 27.2% in the sonographer group (difference of −10.4%, 95% confidence interval: −13.2% to −7.7%, P < 0.001 for non-inferiority, P < 0.001 for superiority). The mean absolute difference between final cardiologist assessment and independent previous cardiologist assessment was 6.29% in the AI group and 7.23% in the sonographer group (difference of −0.96%, 95% confidence interval: −1.34% to −0.54%, P < 0.001 for superiority). The AI-guided workflow saved time for both sonographers and cardiologists, and cardiologists were not able to distinguish between the initial assessments by AI versus the sonographer (blinding index of 0.088). For patients undergoing echocardiographic quantification of cardiac function, initial assessment of LVEF by AI was non-inferior to assessment by sonographers.

Despite rapidly advancing developments in targeted therapeutics and genetic sequencing,^1,2 persistent limits in the accuracy and throughput of clinical phenotyping has led to a widening gap between the potential and the actual benefits realized by precision medicine. This conundrum is exemplified by current approaches to assessing morphologic alterations of the heart.^3,4 If reliably identified, misdiagnoses of certain cardiac diseases (eg, cardiac amyloidosis and hypertrophic cardiomyopathy [HCM]) could be avoided, and specific targeted therapies could be initiated efficiently. Systematic screening paradigms, including through imaging and automated medical record feature review, have shown the opportunity to identify patients with underdiagnosed diseases that are increasingly recognized as more prevalent than was previously thought.^5-9 The ability to reliably distinguish between cardiac disease types with similar morphologic features but different causes would also enhance specificity for linking genetic risk variants and determining mechanisms.

Although physical activity is widely recommended for reducing cardiovascular and all-cause mortality risks, female individuals consistently lag behind male individuals in exercise engagement. The goal of this study was to evaluate whether physical activity derived health benefits may differ by sex. In a prospective study of 412,413 U.S. adults (55% female, age 44 ± 17 years) who provided survey data on leisure-time physical activity, we examined sex-specific multivariable-adjusted associations of physical activity measures (frequency, duration, intensity, type) with all-cause and cardiovascular mortality from 1997 through 2019. During 4,911,178 person-years of follow-up, there were 39,935 all-cause deaths including 11,670 cardiovascular deaths. Regular leisure-time physical activity compared with inactivity was associated with 24% (HR: 0.76; 95% CI: 0.73-0.80) and 15% (HR: 0.85; 95% CI: 0.82-0.89) lower risk of all-cause mortality in women and men, respectively (Wald F = 12.0, sex interaction P < 0.001). Men reached their maximal survival benefit of HR 0.81 from 300 min/wk of moderate-to-vigorous physical activity, whereas women achieved similar benefit at 140 min/wk and then continued to reach a maximum survival benefit of HR 0.76 also at ∼300 min/wk. Sex-specific findings were similar for cardiovascular death (Wald F = 20.1, sex interaction P < 0.001) and consistent across all measures of aerobic activity as well as muscle strengthening activity (Wald F = 6.7, sex interaction P = 0.009). Women compared with men derived greater gains in all-cause and cardiovascular mortality risk reduction from equivalent doses of leisure-time physical activity. These findings could enhance efforts to close the "gender gap" by motivating especially women to engage in any regular leisure-time physical activity.