Primary diabetes care and diabetic retinopathy (DR) screening persist as major public health challenges due to a shortage of trained primary care physicians (PCPs), particularly in low-resource settings. Here, to bridge the gaps, we developed an integrated image–language system (DeepDR-LLM), combining a large language model (LLM module) and image-based deep learning (DeepDR-Transformer), to provide individualized diabetes management recommendations to PCPs. In a retrospective evaluation, the LLM module demonstrated comparable performance to PCPs and endocrinology residents when tested in English and outperformed PCPs and had comparable performance to endocrinology residents in Chinese. For identifying referable DR, the average PCP’s accuracy was 81.0% unassisted and 92.3% assisted by DeepDR-Transformer. Furthermore, we performed a single-center real-world prospective study, deploying DeepDR-LLM. We compared diabetes management adherence of patients under the unassisted PCP arm (n = 397) with those under the PCP+DeepDR-LLM arm (n = 372). Patients with newly diagnosed diabetes in the PCP+DeepDR-LLM arm showed better self-management behaviors throughout follow-up (P < 0.05). For patients with referral DR, those in the PCP+DeepDR-LLM arm were more likely to adhere to DR referrals (P < 0.01). Additionally, DeepDR-LLM deployment improved the quality and empathy level of management recommendations. Given its multifaceted performance, DeepDR-LLM holds promise as a digital solution for enhancing primary diabetes care and DR screening.

Optical coherence tomography angiography (OCT-A) provides depth-resolved visualization of the retinal microvasculature without intravenous dye injection. It facilitates investigations of various retinal vascular diseases and glaucoma by assessment of qualitative and quantitative microvascular changes in the different retinal layers and radial peripapillary layer non-invasively, individually, and efficiently. Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has been applied in OCT-A image analysis in recent years and achieved good performance for different tasks, such as image quality control, segmentation, and classification. DL technologies have further facilitated the potential implementation of OCT-A in eye clinics in an automated and efficient manner and enhanced its clinical values for detecting and evaluating various vascular retinopathies. Nevertheless, the deployment of this combination in real-world clinics is still in the “proof-of-concept” stage due to several limitations, such as small training sample size, lack of standardized data preprocessing, insufficient testing in external datasets, and absence of standardized results interpretation. In this review, we introduce the existing applications of DL in OCT-A, summarize the potential challenges of the clinical deployment, and discuss future research directions.

Advances in artificial intelligence deep learning (DL) have made tremendous impacts on the field of ocular imaging over the last few years. Specifically, DL has been utilised to detect and classify various ocular diseases on retinal photographs, optical coherence tomography (OCT) images, and OCT-angiography images. In order to achieve good robustness and generalisability of model performance, DL training strategies traditionally require extensive and diverse training datasets from various sites to be transferred and pooled into a “centralised location”. However, such a data transferring process could raise practical concerns related to data security and patient privacy. Federated learning (FL) is a distributed collaborative learning paradigm which enables the coordination of multiple collaborators without the need for sharing confidential data. This distributed training approach has great potential to ensure data privacy among different institutions and reduce the potential risk of data leakage from data pooling or centralisation. This review article aims to introduce the concept of FL, provide current evidence of FL in ocular imaging, and discuss potential challenges as well as future applications.

Purpose: To investigate the role of spectral domain optical coherence tomography (SD‐OCT)‐based biomarkers in predicting treatment response of macular oedema (MO) secondary to retinal vein occlusion (RVO) to anti‐vascular endothelial growth factor (VEGF) therapy. Methods: Retrospective cohort study including consecutive cases of RVO associated MO who received anti‐VEGF injections between January 2020 and April 2021. LogMAR visual acuity (VA) at baseline, 12 and 24 months was correlated with a panel of SD‐OCT‐based biomarkers, including vitreomacular status, size of intra‐retinal cysts (IRC), presence of disorganization of retinal inner layers (DRIL), hyper‐reflective foci (HRF) in the retina, integrity of the external limiting membrane (ELM), ellipsoid zone (EZ), cone outer segment tip (COST) and presence of subretinal fluid (SRF). Results: One hundred and thirty eyes were included with 81 and 49 eyes in the BRVO and CRVO subgroup, respectively. In both subgroups, baseline disrupted EZ/ELM [BRVO: ( β = 0.144 p = 0.008; β = 0.111 p = 0.014; β = 0.096 p = 0.042) and CRVO: ( β = 0.316 p < 0.001; β = 0.336 p < 0.001; β = 0.327 p < 0.001)] were associated with worse VA from baseline through 24 months. In the BRVO subgroup, the presence of HRF ( β = 0.209 p < 0.001) correlated with worse baseline VA. Improvement in DRIL extent [OR = 4.355 (1.109–17.094) p = 0.035; OR = 4.510 (1.707–11.917) p = 0.002] and EZ/ELM integrity [OR = 4.474 (1.783–11.223) p = 0.001; OR = 3.214 (1.414–7.305) p = 0.005] were associated with a higher likelihood of achieving at least a 5 letters gain at 12 and 24 months. Conclusion: A comprehensive system of SD‐OCT‐based features could predict functional outcomes of MO secondary to RVO with anti‐VEGF therapy up to 24 months.

Center of pressure (COP) estimation with images/videos as input achieves accurate precision with the development of the human skeleton joint extraction tasks. As a supervised learning task, correct labels acquired from COP with regard to the input images/videos are significant. Thus, synchronization between these two different types of sequences is necessary. If these two different modalities are misaligned, the downstream tasks’ precision is affected significantly due to the inaccurate labels from the COP sequence. In this paper, we used a synchronized dataset and unsupervised deep learning to train an Alignment Network to align video and COP sequences on another unsynchronized dataset where each sequence starts at a different time and has different frame rates. On the synchronized dataset, the Alignment Network removes 84.4% of temporal offset. On the unsynchronized dataset, we proposed a simple yet effective Differential Network to simulate one practical downstream task. We used the differential Network to estimate the sway level of COP. Results show that this method achieved significant improvement (over 20% improvement on three sway level cases) over the misaligned dataset.