Arun James Thirunavukarasu
Auto-generated profile, based on publicly available information

Arun James Thirunavukarasu

International Centre for Eye Health, London School of Hygiene and Tropical Medicine, London, UK

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 July 2023 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
DTDarren Shu Jeng Ting
Darren Shu Jeng Ting
KEKabilan Elangovan
Kabilan Elangovan
LGLaura Gutierrez
Laura Gutierrez
TTTing Fang Tan
Ting Fang Tan
DTDarren Shu Jeng Ting
Darren Shu Jeng Ting

24 July 2023 in Health Care Science
Verified
Journal Article
Open access
RYRui Yang
Rui Yang
TTTing Fang Tan
Ting Fang Tan
WLWei Lu
Wei Lu
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
DTDaniel Shu Wei Ting
Daniel Shu Wei Ting
Nan LiuNan Liu
Nan Liu
Recently, the emergence of ChatGPT, an artificial intelligence chatbot developed by OpenAI, has attracted significant attention due to its exceptional language comprehension and content generation capabilities, highlighting the immense potential of large language models (LLMs). LLMs have become a burgeoning hotspot across many fields, including health care. Within health care, LLMs may be classified into LLMs for the biomedical domain and LLMs for the clinical domain based on the corpora used for pre‐training. In the last 3 years, these domain‐specific LLMs have demonstrated exceptional performance on multiple natural language processing tasks, surpassing the performance of general LLMs as well. This not only emphasizes the significance of developing dedicated LLMs for the specific domains, but also raises expectations for their applications in health care. We believe that LLMs may be used widely in preconsultation, diagnosis, and management, with appropriate development and supervision. Additionally, LLMs hold tremendous promise in assisting with medical education, medical writing and other related applications. Likewise, health care systems must recognize and address the challenges posed by LLMs.

21 April 2023 in JMIR Medical Education
Verified
Journal Article
Open access
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
RHRefaat Hassan
Refaat Hassan
SMShathar Mahmood
Shathar Mahmood
RSRohan Sanghera
Rohan Sanghera
KBKara Barzangi
Kara Barzangi
SSSachin Shah
Sachin Shah
Background: Large language models exhibiting human-level performance in specialized tasks are emerging; examples include Generative Pretrained Transformer 3.5, which underlies the processing of ChatGPT. Rigorous trials are required to understand the capabilities of emerging technology, so that innovation can be directed to benefit patients and practitioners. Objective: Here, we evaluated the strengths and weaknesses of ChatGPT in primary care using the Membership of the Royal College of General Practitioners Applied Knowledge Test (AKT) as a medium. Methods: AKT questions were sourced from a web-based question bank and 2 AKT practice papers. In total, 674 unique AKT questions were inputted to ChatGPT, with the model’s answers recorded and compared to correct answers provided by the Royal College of General Practitioners. Each question was inputted twice in separate ChatGPT sessions, with answers on repeated trials compared to gauge consistency. Subject difficulty was gauged by referring to examiners’ reports from 2018 to 2022. Novel explanations from ChatGPT—defined as information provided that was not inputted within the question or multiple answer choices—were recorded. Performance was analyzed with respect to subject, difficulty, question source, and novel model outputs to explore ChatGPT’s strengths and weaknesses. Results: Average overall performance of ChatGPT was 60.17%, which is below the mean passing mark in the last 2 years (70.42%). Accuracy differed between sources (P=.04 and .06). ChatGPT’s performance varied with subject category (P=.02 and .02), but variation did not correlate with difficulty (Spearman ρ=–0.241 and –0.238; P=.19 and .20). The proclivity of ChatGPT to provide novel explanations did not affect accuracy (P>.99 and .23). Conclusions: Large language models are approaching human expert–level performance, although further development is required to match the performance of qualified primary care physicians in the AKT. Validated high-performance models may serve as assistants or autonomous clinical tools to ameliorate the general practice workforce crisis.

6 October 2025 in Nature Medicine
Verified
Journal Article
Closed access
PDF file
ZTZhen Ling Teo
Zhen Ling Teo
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
KEKabilan Elangovan
Kabilan Elangovan
HCHaoran Cheng
Haoran Cheng
PMPrasanth Moova
Prasanth Moova
DTDarren Shu Jeng Ting
Darren Shu Jeng Ting

17 April 2024 in PLOS Digital Health
Verified
Journal Article
Open access
PDF file
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
SMShathar Mahmood
Shathar Mahmood
AMAndrew Malem
Andrew Malem
WFWilliam Paul Foster
William Paul Foster
RSRohan Sanghera
Rohan Sanghera
DTDarren Shu Jeng Ting
Darren Shu Jeng Ting
Large language models (LLMs) underlie remarkable recent advanced in natural language processing, and they are beginning to be applied in clinical contexts. We aimed to evaluate the clinical potential of state-of-the-art LLMs in ophthalmology using a more robust benchmark than raw examination scores. We trialled GPT-3.5 and GPT-4 on 347 ophthalmology questions before GPT-3.5, GPT-4, PaLM 2, LLaMA, expert ophthalmologists, and doctors in training were trialled on a mock examination of 87 questions. Performance was analysed with respect to question subject and type (first order recall and higher order reasoning). Masked ophthalmologists graded the accuracy, relevance, and overall preference of GPT-3.5 and GPT-4 responses to the same questions. The performance of GPT-4 (69%) was superior to GPT-3.5 (48%), LLaMA (32%), and PaLM 2 (56%). GPT-4 compared favourably with expert ophthalmologists (median 76%, range 64–90%), ophthalmology trainees (median 59%, range 57–63%), and unspecialised junior doctors (median 43%, range 41–44%). Low agreement between LLMs and doctors reflected idiosyncratic differences in knowledge and reasoning with overall consistency across subjects and types (p>0.05). All ophthalmologists preferred GPT-4 responses over GPT-3.5 and rated the accuracy and relevance of GPT-4 as higher (p<0.05). LLMs are approaching expert-level knowledge and reasoning skills in ophthalmology. In view of the comparable or superior performance to trainee-grade ophthalmologists and unspecialised junior doctors, state-of-the-art LLMs such as GPT-4 may provide useful medical advice and assistance where access to expert ophthalmologists is limited. Clinical benchmarks provide useful assays of LLM capabilities in healthcare before clinical trials can be designed and conducted.

22 March 2025 in Journal of the American Medical Informatics Association
Verified
Journal Article
Open access
RSRohan Sanghera
Rohan Sanghera
ATArun James Thirunavukarasu
Arun James Thirunavukarasu
MKMarc El Khoury
Marc El Khoury
JOJessica O’logbon
Jessica O’logbon
YCYuqing Chen
Yuqing Chen
ASAndrew A S Soltan
Andrew A S Soltan
Objective: Abstract screening is a labor-intensive component of systematic review involving repetitive application of inclusion and exclusion criteria on a large volume of studies. We aimed to validate large language models (LLMs) used to automate abstract screening. Materials and Methods: LLMs (GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o, Llama 3 70B, Gemini 1.5 Pro, and Claude Sonnet 3.5) were trialed across 23 Cochrane Library systematic reviews to evaluate their accuracy in zero-shot binary classification for abstract screening. Initial evaluation on a balanced development dataset (n = 800) identified optimal prompting strategies, and the best performing LLM-prompt combinations were then validated on a comprehensive dataset of replicated search results (n = 119 695). Results: On the development dataset, LLMs exhibited superior performance to human researchers in terms of sensitivity (LLMmax = 1.000, humanmax = 0.775), precision (LLMmax = 0.927, humanmax = 0.911), and balanced accuracy (LLMmax = 0.904, humanmax = 0.865). When evaluated on the comprehensive dataset, the best performing LLM-prompt combinations exhibited consistent sensitivity (range 0.756-1.000) but diminished precision (range 0.004-0.096) due to class imbalance. In addition, 66 LLM-human and LLM-LLM ensembles exhibited perfect sensitivity with a maximal precision of 0.458 with the development dataset, decreasing to 0.1450 over the comprehensive dataset; but conferring workload reductions ranging between 37.55% and 99.11%. Discussion: Automated abstract screening can reduce the screening workload in systematic review while maintaining quality. Performance variation between reviews highlights the importance of domain-specific validation before autonomous deployment. LLM-human ensembles can achieve similar benefits while maintaining human oversight over all records. Conclusion: LLMs may reduce the human labor cost of systematic review with maintained or improved accuracy, thereby increasing the efficiency and quality of evidence synthesis.