Background Large language models (LLMs), such as ChatGPT, have considerable implications for various medical applications. However, ChatGPT’s training primarily draws from English-centric internet data and is not tailored explicitly to the medical domain. Thus, an ophthalmic LLM in Chinese is clinically essential for both healthcare providers and patients in mainland China. Methods We developed an LLM of ophthalmology (MOPH) using Chinese corpora and evaluated its performance in three clinical scenarios: ophthalmic board exams in Chinese, answering evidence-based medicine-oriented ophthalmic questions and diagnostic accuracy for clinical vignettes. Additionally, we compared MOPH’s performance to that of human doctors. Results In the ophthalmic exam, MOPH’s average score closely aligned with the mean score of trainees (64.7 (range 62–68) vs 66.2 (range 50–92), p=0.817), but achieving a score above 60 in all seven mock exams. In answering ophthalmic questions, MOPH demonstrated an adherence of 83.3% (25/30) of responses following Chinese guidelines (Likert scale 4–5). Only 6.7% (2/30, Likert scale 1–2) and 10% (3/30, Likert scale 3) of responses were rated as ‘poor or very poor’ or ‘potentially misinterpretable inaccuracies’ by reviewers. In diagnostic accuracy, although the rate of correct diagnosis by ophthalmologists was superior to that by MOPH (96.1% vs 81.1%, p>0.05), the difference was not statistically significant. Conclusion This study demonstrated the promising performance of MOPH, a Chinese-specific ophthalmic LLM, in diverse clinical scenarios. MOPH has potential real-world applications in Chinese-language ophthalmology settings.
Background: The purpose of this study was to implement and evaluate a deep learning (DL) approach for automatically detecting shallow anterior chamber depth (ACD) from two-dimensional (2D) overview anterior segment photographs. Methods: We trained a DL model using a dataset of anterior segment photographs collected from Shanghai Aier Eye Hospital from June 2018 to December 2019. A Pentacam HR system was used to capture a 2D overview eye image and measure the ACD. Shallow ACD was defined as ACD less than 2.4 mm. The DL model was evaluated by a five-fold cross-validation test in a hold-out testing dataset. We also evaluated the DL model by testing it against two glaucoma specialists. The performance of the DL model was calculated by metrics, including accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Results: A total of 3753 photographs (1720 shallow AC and 2033 deep AC images) were assigned to the training dataset, and 1302 photographs (509 shallow AC and 793 deep AC images) were held out for two internal testing datasets. In detecting shallow ACD in the internal hold-out testing dataset, the DL model achieved an AUC of 0.86 (95% CI, 0.83–0.90) with 80% sensitivity and 79% specificity. In the same testing dataset, the DL model also achieved better performance than the two glaucoma specialists (accuracy of 80% vs. accuracy of 74 and 69%). Conclusions: We proposed a high-performing DL model to automatically detect shallow ACD from overview anterior segment photographs. Our DL model has potential applications in detecting and monitoring shallow ACD in the real world. Trial registration: http://clinicaltrials.gov, NCT04340635, retrospectively registered on 29 March 2020.
BACKGROUND Macular diseases are one of the leading causes of blindness. Diagnosis of macular diseases is an important part of ophthalmic education. Optical coherence tomography (OCT) is now a standard imaging modality for diagnosing and treating macular disease. Accurate interpretations of OCT with macular diseases are vital for ophthalmic resident training worldwide. Previously, we proposed generative adversarial networks (GANs) model for synthetic OCT images of macular disorders. Our result also showed its potential value for educational purposes. This study aimed to establish an educational tool based on GANs synthetic images for ophthalmic residents’ and medical students’ education and evaluate its validity. OBJECTIVE To evaluate the assessment of GANs synthetic OCT images as an education tool for image diagnosis of macular diseases in medical students and ophthalmic residents trainingIn Group GANs OCT, scores improved from 80.0(46.0, 85.5) to 92.0(81.0,95.5) five minutes after training (p<0.001) and 92.30± 5.36 one-week after training (p<0.001). Similarly, in Group real OCT, scores improved from 66.00±19.52 to 92.90± 5.71(p<0.001) and 93.20± 6.10 (p<0.001), respectively. No significantly difference were found for scores of tests, score improvements and time consumptions between Group GANs OCT and Group real OCT. Medical students had significantly higher score improvement after training than residents(p<0.001) did. METHODS In this randomized trial, twenty fifth-year medical students and 20 ophthalmic residents were enrolled and randomly assigned (1:1 allocation) into Group real OCT and Group GANs OCT. All participants had a pre-test to show their educational background, followed by a 30-minute-smartphone-based GANs/real OCT images-based education program for macular diseases recognition training. Another two tests were scheduled five minutes and one week after the training to test their short-term and long-term performances. Scores and time consumptions were recorded and compared. After all tests, an anonymous subjective questionnaire was acquired. RESULTS In Group GANs OCT, scores improved from 80.0(46.0, 85.5) to 92.0(81.0,95.5) five minutes after training (p<0.001) and 92.30± 5.36 one-week after training (p<0.001). Similarly, in Group real OCT, scores improved from 66.00±19.52 to 92.90± 5.71(p<0.001) and 93.20± 6.10 (p<0.001), respectively. No significantly difference were found for scores of tests, score improvements and time consumptions between Group GANs OCT and Group real OCT. Medical students had significantly higher score improvement after training than residents(p<0.001) did. CONCLUSIONS The education tool using synthetic OCT images had a similar educational ability compared to that using real OCT images, which improved the interpretation ability of ophthalmic residents and medical students in both short-term and long-term performances. The smartphone-based educational tool could be promoted more widely for educational applications. CLINICALTRIAL Chinses Clinical Trial Registry, No. ChiCTR 2100053195.