Background: Large language models exhibiting human-level performance in specialized tasks are emerging; examples include Generative Pretrained Transformer 3.5, which underlies the processing of ChatGPT. Rigorous trials are required to understand the capabilities of emerging technology, so that innovation can be directed to benefit patients and practitioners. Objective: Here, we evaluated the strengths and weaknesses of ChatGPT in primary care using the Membership of the Royal College of General Practitioners Applied Knowledge Test (AKT) as a medium. Methods: AKT questions were sourced from a web-based question bank and 2 AKT practice papers. In total, 674 unique AKT questions were inputted to ChatGPT, with the model’s answers recorded and compared to correct answers provided by the Royal College of General Practitioners. Each question was inputted twice in separate ChatGPT sessions, with answers on repeated trials compared to gauge consistency. Subject difficulty was gauged by referring to examiners’ reports from 2018 to 2022. Novel explanations from ChatGPT—defined as information provided that was not inputted within the question or multiple answer choices—were recorded. Performance was analyzed with respect to subject, difficulty, question source, and novel model outputs to explore ChatGPT’s strengths and weaknesses. Results: Average overall performance of ChatGPT was 60.17%, which is below the mean passing mark in the last 2 years (70.42%). Accuracy differed between sources (P=.04 and .06). ChatGPT’s performance varied with subject category (P=.02 and .02), but variation did not correlate with difficulty (Spearman ρ=–0.241 and –0.238; P=.19 and .20). The proclivity of ChatGPT to provide novel explanations did not affect accuracy (P>.99 and .23). Conclusions: Large language models are approaching human expert–level performance, although further development is required to match the performance of qualified primary care physicians in the AKT. Validated high-performance models may serve as assistants or autonomous clinical tools to ameliorate the general practice workforce crisis.
Introduction: Cost-effective interventions that improve medication adherence are urgently needed to address the epidemic of non-communicable diseases (NCDs) in India. However, in low- and middle-income countries like India, there is a lack of analysis evaluating the effectiveness of adherence improving strategies. We conducted the first systematic review evaluating interventions aimed at improving medication adherence for chronic diseases in India.Methods: A systematic search on MEDLINE, Web of Science, Scopus, and Google Scholar was conducted. Based on a PRISMA-compliant, pre-defined methodology, randomized control trials were included which: involved subjects with NCDs; were located in India; used any intervention with the aim of improving medication adherence; and measured adherence as a primary or secondary outcome.Results: The search strategy yielded 1,552 unique articles of which 22 met inclusion criteria. Interventions assessed by these studies included education-based interventions (n = 12), combinations of education-based interventions with regular follow up (n = 4), and technology-based interventions (n = 2). Non-communicable diseases evaluated commonly were respiratory disease (n = 3), type 2 diabetes (n = 6), cardiovascular disease (n = 8) and depression (n = 2).Conclusions: Although the vast majority of primary studies supporting the conclusions were of mixed methodological quality, patient education by CHWs and pharmacists represent promising interventions to improve medication adherence, with further benefits from regular follow-up. There is need for systematic evaluation of these interventions with high quality RCTs and their implementation as part of wider health policy.Systematic review registration: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022345636, identifier: CRD42022345636.
Introduction Metabolic syndrome is a significance driver of mortality and morbidity in India, but nationally representative data regarding disease burden and treatment status are lacking. Here, a cross-sectional study was undertaken to establish national and regional estimates of disease burden and explore reasons for lack of treatment of component conditions of metabolic syndrome in Indian older adults (45 years and older). Methods A cross-sectional study was undertaken using data from the first wave of the Longitudinal Ageing Study in India (2017–2018). Data for 66 606 individuals aged 45 years and above were analysed. The primary outcome was metabolic syndrome prevalence, defined by the National Cholesterol Education Programme ATP III criteria as an individual having any three of four component conditions: diabetes mellitus (DM), hypertension, abdominal obesity and hypercholesterolaemia. The secondary outcome of this study was treatment status of patients with component conditions. Results Metabolic syndrome was found to have an overall weighted prevalence of 4.83% (n=3630, 95% CI 4.24 to 5.51). Females, urban residents, obese individuals and physically inactive people exhibited greater prevalence. The most prevalent component of metabolic syndrome was hypertension followed by abdominal obesity, DM and hypercholesterolaemia. 8.85% metabolic syndrome patient reported no treatment for component conditions, while 17.58% reported only partial treatment. Elderly individuals between 60 and 69 (crude relative risk ratios, cRRR 2.20, 95% CI 1.20 to 4.01) and 80 years and above (cRRR 7.48, 95% CI 1.99 to 28.16), urban residents (cRRR 2.45, 95% CI 1.48 to 4.05), those from richer monthly per capita consumption expenditure quintiles (cRRR 2.55, 95% CI 1.00 to 6.47) and those with additional comorbidities (cRRR 2.17, 95% CI 1.28 to 3.70) were more likely to report comprehensive treatment. Conclusions This study highlights a substantial prevalence of metabolic syndrome in older adults in India and reveals remarkable disparities in provision of treatment. Better prevention, earlier detection and improved provision of treatment are urgently required to combat the rising prevalence of metabolic syndrome and reduce the burden of cardiovascular disease in India.
BACKGROUND Large language models exhibiting human-level performance in specialized tasks are emerging; examples include Generative Pretrained Transformer 3.5, which underlies the processing of ChatGPT. Rigorous trials are required to understand the capabilities of emerging technology, so that innovation can be directed to benefit patients and practitioners. OBJECTIVE Here, we evaluated the strengths and weaknesses of ChatGPT in primary care using the Membership of the Royal College of General Practitioners Applied Knowledge Test (AKT) as a medium. METHODS AKT questions were sourced from a web-based question bank and 2 AKT practice papers. In total, 674 unique AKT questions were inputted to ChatGPT, with the model’s answers recorded and compared to correct answers provided by the Royal College of General Practitioners. Each question was inputted twice in separate ChatGPT sessions, with answers on repeated trials compared to gauge consistency. Subject difficulty was gauged by referring to examiners’ reports from 2018 to 2022. Novel explanations from ChatGPT—defined as information provided that was not inputted within the question or multiple answer choices—were recorded. Performance was analyzed with respect to subject, difficulty, question source, and novel model outputs to explore ChatGPT’s strengths and weaknesses. RESULTS Average overall performance of ChatGPT was 60.17%, which is below the mean passing mark in the last 2 years (70.42%). Accuracy differed between sources (P=.04 and .06). ChatGPT’s performance varied with subject category (P=.02 and .02), but variation did not correlate with difficulty (Spearman ρ=–0.241 and –0.238; P=.19 and .20). The proclivity of ChatGPT to provide novel explanations did not affect accuracy (P>.99 and .23). CONCLUSIONS Large language models are approaching human expert–level performance, although further development is required to match the performance of qualified primary care physicians in the AKT. Validated high-performance models may serve as assistants or autonomous clinical tools to ameliorate the general practice workforce crisis.