It is likely that individuals are turning to Large Language Models (LLMs) to seek health advice, much like searching for diagnoses on Google. We evaluate clinical accuracy of GPT-3·5 and GPT-4 for suggesting initial diagnosis, examination steps and treatment of 110 medical cases across diverse clinical disciplines. Moreover, two model configurations of the Llama 2 open source LLMs are assessed in a sub-study. For benchmarking the diagnostic task, we conduct a naïve Google search for comparison. Overall, GPT-4 performed best with superior performances over GPT-3·5 considering diagnosis and examination and superior performance over Google for diagnosis. Except for treatment, better performance on frequent vs rare diseases is evident for all three approaches. The sub-study indicates slightly lower performances for Llama models. In conclusion, the commercial LLMs show growing potential for medical question answering in two successive major releases. However, some weaknesses underscore the need for robust and regulated AI models in health care. Open source LLMs can be a viable option to address specific needs regarding data privacy and transparency of training.

The utilisation of smart devices, such as smartwatches and smartphones, in the field of movement disorders research has gained significant attention. However, the absence of a comprehensive dataset with movement data and clinical annotations, encompassing a wide range of movement disorders including Parkinson’s disease (PD) and its differential diagnoses (DD), presents a significant gap. The availability of such a dataset is crucial for the development of reliable machine learning (ML) models on smart devices, enabling the detection of diseases and monitoring of treatment efficacy in a home-based setting. We conducted a three-year cross-sectional study at a large tertiary care hospital. A multi-modal smartphone app integrated electronic questionnaires and smartwatch measures during an interactive assessment designed by neurologists to provoke subtle changes in movement pathologies. We captured over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC). After age-matching, an integrative ML approach combining classical signal processing and advanced deep learning techniques was implemented and cross-validated. The models achieved an average balanced accuracy of 91.16% in the classification PD vs. HC, while PD vs. DD scored 72.42%. The numbers suggest promising performance while distinguishing similar disorders remains challenging. The extensive annotations, including details on demographics, medical history, symptoms, and movement steps, provide a comprehensive database to ML techniques and encourage further investigations into phenotypical biomarkers related to movement disorders.

Machine learning algorithms become increasingly prevalent in the field of medicine, as they offer the ability to recognize patterns in complex medical data. Especially in this sensitive area, the active usage of a mostly black box is a controversial topic. We aim to highlight how an aggregated and systematic feature analysis of such models can be beneficial in the medical context. For this reason, we introduce a grouped version of the permutation importance analysis for evaluating the influence of entire feature subsets in a machine learning model. In this way, expert-defined subgroups can be evaluated in the decision-making process. Based on these results, new hypotheses can be formulated and examined.

Background and Aims: Identification of patients with acute coronary syndrome requiring coronary revascularization can be challenging due to inconclusive electrocardiogram (ECG) findings or biomarker results. A deep learning model to detect ECG patterns associated with revascularization likelihood was developed, aiming to guide further assessment and reduce diagnostic uncertainty. Methods: A convolutional neural network model was trained on 144 691 ED visits from a US cohort (60 ± 19 years; 53% female; 0.6% revascularization), tested on a separate test cohort (n = 35 995), and benchmarked against clinician ECG interpretation and cardiac troponin T (TnT). External validation was performed for the outcomes revascularization and Type 1 myocardial infarction (MI) on 18 673 ED visits from Europe (55 ± 21 years; 49% female; 1.5% revascularization; 1% Type 1 MI). Primary performance metric was area under the receiver operating characteristic curve (AUROC). Results: In the test cohort, the model achieved an AUROC of 0.91 (95% confidence interval [CI] 0.91–0.91), outperforming clinician ECG interpretation (AUROC 0.65, 95% CI 0.54–0.76) and conventional cardiac TnT (AUROC 0.71). In the external validation cohort, ECG model AUROC was 0.81 (95% CI 0.81–0.82) for revascularization, and 0.85 (95% CI 0.84–0.85) for Type 1 MI, compared with 0.70 (95% CI 0.57–0.83) and 0.74 (95% CI 0.56–0.92) for clinician interpretation, and 0.85 and 0.87 for high-sensitivity (hs)-TnT, respectively. The ECG model had higher specificity but lower sensitivity compared with high-sensitivity-troponin T. Conclusions: The model was able to detect revascularization and Type 1 MI with competitive performance, suggesting a potential role to complement current clinical assessment.

Deep Learning architectures for time series require a large number of training samples, however traditional sample size estimation for sufficient model performance is not applicable for machine learning, especially in the field of electrocardiograms (ECGs). This paper outlines a sample size estimation strategy for binary classification problems on ECGs using different deep learning architectures and the large publicly available PTB-XL dataset, which includes 21801 ECG samples. This work evaluates binary classification tasks for Myocardial Infarction (MI), Conduction Disturbance (CD), ST/T Change (STTC), and Sex. All estimations are benchmarked across different architectures, including XResNet, Inception-, XceptionTime and a fully convolutional network (FCN). The results indicate trends for required sample sizes for given tasks and architectures, which can be used as orientation for future ECG studies or feasibility aspects.

Aims: Automated QTc measurements from commercial ECG systems often diverge from expert readings. We developed QTcNet, a deep learning model trained and validated on multiple large ECG datasets to improve automated QTc measurement accuracy. Methods and results: QTcNet employs a regression-based convolutional neural network architecture. It was trained on 120 300 algorithm-labelled ECGs (60 150 from an internal hospital cohort and 60 150 from the MIMIC-IV dataset) after correction for a vendor-specific +15 ms bias. Performance was evaluated against expert QTc measurements in three independent datasets: PTB Diagnostic ECG Database (n = 100 ECGs in validation set), QTcMS (n = 210), and ECGRDVQ (n = 5219). The effect of fine-tuning on cardiologist-annotated ECGs was tested in the PTB database (n = 449 in fine-tuning set). Model explainability analyses were performed with Integrated Gradient maps. QTcNet reduced cross-cohort mean absolute error (MAE) from 23.4 to 13.4 ms and root mean square error (RMSE) from 40.1 to 22.1 ms, almost halving large (>50 ms) outliers. Fine-tuning only reduced errors in the PTB dataset but did not improve cross-cohort performance. Integrated Gradient maps confirmed that the model concentrated on QRS onset and T wave offset, supporting physiological plausibility. Conclusion: QTcNet, trained on large-scale algorithmically labelled data, consistently outperformed conventional algorithms across three independent, external validation datasets. Fine-tuning of QTcNet may adapt the model to the characteristics of specific cohorts but reduces external validity in other cohorts. We openly release the full model and code, along with a ready-to-use online implementation at https://qtcnet.uni-muenster.de, facilitating further research and community-driven improvement.