Purpose: This study compares computed tomography angiography-based collateral scoring systems in regard to their inter-rater reliability and potential to predict functional outcome after endovascular thrombectomy, and relates them to parenchymal perfusion as measured by computed tomography perfusion. Methods: Eighty-four patients undergoing endovascular thrombectomy in anterior circulation ischaemic stroke were enrolled. Modified Tan Score, Miteff Score, Maas Score and Opercular Index Score ratio were assessed in pre-interventional computed tomography angiographies independently by two readers. Collateral scores were tested for inter-rater reliability by weighted-kappa, for correlations with three-months modified Rankin Scale, and their potential to differentiate between patients with favourable (modified Rankin Scale ≤2) and poor outcome (modified Rankin Scale ≥3). Correlations with relative cerebral blood volume and relative cerebral blood flow were tested in patients with available computed tomography perfusion. Results: Very good inter-rater reliability was found for Modified Tan, Miteff and Opercular Index Score ratio, and substantial reliability for Maas. There were no significant correlations between collateral scores and three-months modified Rankin Scale, but significant group differences between patients with favourable and poor outcome for Maas, Miteff and Opercular Index Score ratio. Miteff and Maas were significant predictors of favourable outcome in binary logistic regression analysis. Miteff best differentiated between both outcome groups in receiver-operating characteristics, and Maas reached highest sensitivity for favourable outcome prediction of 96%. All collateral scores significantly correlated with mean relative cerebral blood volume and relative cerebral blood flow. Conclusions: Computed tomography angiography scores are valuable in estimating functional outcome after mechanical thrombectomy and reliable across readers. The more complex scores, Maas and Miteff, show the best performances in predicting favourable outcome.
Purpose: To investigate the feasibility of Retrieval-augmented Generation (RAG)-enhanced Large Language Models (LLMs) in answering questions about two German neurovascular guidelines. Methods: Four LLMs (GPT-4o-mini, Llama 3.1 405B Instruct Turbo, Mixtral 8 × 22B Instruct, and Claude 3.5 Sonnet) with RAG as well as GPT-4o-mini without RAG were evaluated for generating answers about two German neurovascular guidelines (“S3 Guideline for Diagnosis, Treatment, and Follow-up of Extracranial Carotid Stenosis” and “S2e Guideline for Acute Therapy of Ischemic Stroke”). The answers were classified as “correct”, “inaccurate”, or “incorrect” by two neurovascular experts in consensus. Additionally, retrieval performance of five retrieval strategies was analyzed on a synthetic dataset of 384 questions. Results: Claude Sonnet 3.5 achieved the highest answer correctness (70.6% correct, 10.6% wrong), followed by Llama 3.1 (64.7%, 15.3% wrong), GPT-4o-mini with RAG (57.6%, 15.3% wrong), and Mixtral (56.6%, 17.6% wrong). GPT-4o-mini without RAG performed significantly worse (20.0%, 32.9% wrong). Retrieval errors were the primary cause of incorrect answers (80%). For retrieval, BM25 achieved the highest accuracy (82.0%), outperforming vector-based methods like “BAAI/bge-m3” (78.4%). Conclusion: RAG significantly improves LLM accuracy for medical guideline question answering compared to the inherent knowledge of pretrained LLMs alone while still showing significant error rates. Improved accuracy and confidence metrics are needed for safer implementation in clinical routine. Additionally, our results demonstrate the strong performance of general LLMs in medical question answering for non-English languages, such as German, even without specific training.
PurposeTwo strategies of initial patient care exist in endovascular thrombectomy (ET) depending on the site of initial admission: the mothership (MS) and drip-and-ship (DnS) principles. This study compares both strategies in regard to patient outcome in a local network of specialized hospitals.MethodsTwo-hundred-and-two patients undergoing ET in anterior circulation ischemic stroke between June 2016 and May 2018 were enrolled. Ninety two patients were directly admitted to our local facility (MS), One-hundred-and-ten were secondarily referred to our facility. Group comparisons between admission strategies in three-months modified Rankin Scale (mRS), Maas Score and Alberta-Stroke-Program-Early-computed-tomography-score (ASPECTS), National-Institutes-of-Health-Stroke-Scale (NIHSS), age and onset-to-recanalization-time were performed. Correlation between admission strategy and mRS was calculated. A binary logistic regression model was computed including mRS as dependent variable.ResultsThere were neither significant group differences in three-months mRS between MS and DnS nor significant correlations. Patients tended to achieve a better outcome with DnS. Collateralization status differed between MS and DnS (p = 0.003) with better collateralization in DnS. There were no significant group differences in NIHSS or ASPECTS but in onset-to-recanalization-time (p < 0.001) between MS and DnS. Binary logistic regression showed a high explanation of variance of mRS but no significant results for admission strategy.ConclusionsFunctional outcome in patients treated with ET is comparable between the MS and DnS principles. Tendentially better outcome in the DnS subgroup may be explained by selection bias due to a higher willingness to apply ET in patients with worse baseline conditions (e.g. worse collateralization), if patients undergoing MS are already on site.