4 November 2020 in Journal of Open Source Software
27 May 2021 in International Journal of Computer Assisted Radiology and Surgery
Purpose: Registration of Laparoscopic Ultrasound (LUS) to a pre-operative scan such as Computed Tomography (CT) using blood vessel information has been proposed as a method to enable image-guidance for laparoscopic liver resection. Currently, there are solutions for this problem that can potentially enable clinical translation by bypassing the need for a manual initialisation and tracking information. However, no reliable framework for the segmentation of vessels in 2D untracked LUS images has been presented. Methods: We propose the use of 2D UNet for the segmentation of liver vessels in 2D LUS images. We integrate these results in a previously developed registration method, and show the feasibility of a fully automatic initialisation to the LUS to CT registration problem without a tracking device. Results: We validate our segmentation using LUS data from 6 patients. We test multiple models by placing patient datasets into different combinations of training, testing and hold-out, and obtain mean Dice scores ranging from 0.543 to 0.706. Using these segmentations, we obtain registration accuracies between 6.3 and 16.6 mm in 50% of cases. Conclusions: We demonstrate the first instance of deep learning (DL) for the segmentation of liver vessels in LUS. Our results show the feasibility of UNet in detecting multiple vessel instances in 2D LUS images, and potentially automating a LUS to CT registration pipeline.
13 September 2024 in medRxiv
Integrating large language models (LLMs) into healthcare settings can improve workflow efficiency and patient care by automating tasks such as summarising consultations. However, ensuring the fidelity between LLM outputs and ground truth information is crucial, as errors can lead to miscommunication between patients and clinicians, resulting in incorrect diagnoses, treatment decisions and compromised patient safety. We introduce a clinician-in-the-loop framework with: 1) a clinically and technically-informed error taxonomy to classify LLM outputs, 2) an experimental structure to comprehensively and iteratively compare outputs within our LLM document generation pipeline, 3) a clinical safety framework to assess potential harms of errors in LLM outputs, and 4) an encompassing graphical user interface (GUI), CREOLA, to perform and assess all previous steps. Our clinical error metrics were derived from 18 experimental configurations involving LLMs for clinical note generation consisting of 49,590 transcript and 12,999 clinical note sentences. Overall, we observed a 1.47% hallucination rate (44% rated ‘major’) and a 3.45% omission rate (17% ‘major’). Through iterative prompts and workflow refinements, we reduced major errors below previously reported human note-taking error rates, underscoring the potential of our framework to enable safer clinical documentation.
16 May 2023 in Journal of Open Source Software
16 April 2024 in Medical Image Analysis
Supervised machine learning-based medical image computing applications necessitate expert label curation, while unlabelled image data might be relatively abundant. Active learning methods aim to prioritise a subset of available image data for expert annotation, for label-efficient model training. We develop a controller neural network that measures priority of images in a sequence of batches, as in batch-mode active learning, for multi-class segmentation tasks. The controller is optimised by rewarding positive task-specific performance gain, within a Markov decision process (MDP) environment that also optimises the task predictor. In this work, the task predictor is a segmentation network. A meta-reinforcement learning algorithm is proposed with multiple MDPs, such that the pre-trained controller can be adapted to a new MDP that contains data from different institutes and/or requires segmentation of different organs or structures within the abdomen. We present experimental results using multiple CT datasets from more than one thousand patients, with segmentation tasks of nine different abdominal organs, to demonstrate the efficacy of the learnt prioritisation controller function and its cross-institute and cross-organ adaptability. We show that the proposed adaptable prioritisation metric yields converging segmentation accuracy for a new kidney segmentation task, unseen in training, using between approximately 40% to 60% of labels otherwise required with other heuristic or random prioritisation metrics. For clinical datasets of limited size, the proposed adaptable prioritisation offers a performance improvement of 22.6% and 10.2% in Dice score, for tasks of kidney and liver vessel segmentation, respectively, compared to random prioritisation and alternative active sampling strategies.