The coronavirus disease 2019 (COVID-19) has become a severe worldwide health emergency and is spreading at a rapid rate. Segmentation of COVID lesions from computed tomography (CT) scans is of great importance for supervising disease progression and further clinical treatment. As labeling COVID-19 CT scans is labor-intensive and time-consuming, it is essential to develop a segmentation method based on limited labeled data to conduct this task. In this paper, we propose a self-ensembled co-training framework, which is trained by limited labeled data and large-scale unlabeled data, to automatically extract COVID lesions from CT scans. Specifically, to enrich the diversity of unsupervised information, we build a co-training framework consisting of two collaborative models, in which the two models teach each other during training by using their respective predicted pseudo-labels of unlabeled data. Moreover, to alleviate the adverse impacts of noisy pseudo-labels for each model, we propose a self-ensembling strategy to perform consistency regularization for the up-to-date predictions of unlabeled data, in which the predictions of unlabeled data are gradually ensembled via moving average at the end of every training epoch. We evaluate our framework on a COVID-19 dataset containing 103 CT scans. Experimental results show that our proposed method achieves better performance in the case of only 4 labeled CT scans compared to the state-of-the-art semi-supervised segmentation networks.

Thermal ablation is an exciting new minimally invasive treatment that destroys liver tumors without removing them. It uses image guidance to place a needle through the skin into a liver tumor, which is highly dependent on surgeons’ experience. With the development of digital medicine, augmented reality (AR) has become a more intuitive and safer way to achieve real-time navigation. However, the technology is still in its infancy due to its limited accuracy and real-time performance. To address these problems, we syncretized the holographic AR with the digital twin technique to track the dynamic surgical scene and provide the 3-D navigation of heterogeneous target regions via internal motion prediction. To tackle the dilemma of real-time performance and precise internal motion estimation, a dynamic adaptation scheme is proposed to compensate for the time cost induced by the external/internal correlation model and data transmission. We carried out a series of experiments to validate our methods. With the proposed external/internal correlation model, the average estimation errors of the tumor and vessels are 2.18 and 2.79 mm, respectively. Besides, we performed in vivo experiments on two beagle dogs with an artificial lesion in their liver, respectively, and the puncture accuracy of our method are 2.5 and 2.17 mm. The results show that on one hand, our method can fulfill the real-time requirement of AR via using the intraoperative data, which is also more precise than that with preoperative data. On the other hand, our method can provide more 3-D information for surgeons, such as vessels, which can well ensure the safety of operation.

This paper presents a novel mixed reality (MR) guidance method for liver tumours radiofrequency ablation (RFA). Compared with traditional CT-guided method, our system can provide a more natural and intuitive surgical mode for surgeons. In essence, our system is a holographic navigation platform, which projects a mixed-reality overlay onto the patient via HoloLens during RFA. We first reconstruct the patient-specific anatomy structure from the CT images of abdominal phantom. Then, a tailored precise registration method is employed to map the virtual-real spatial information. In addition, considering that tumor shifting during biopsy severely impacts the accuracy of RFA, our guidance system involves a motion compensation computation through data-driven physicallybased modeling in holographic environment. In experiments, we conduct a user study on the comparison trial between MR-guided and CT-guided biopsy. User feedback demonstrates that our MR guidance method for needle placement procedure has the potential to simplify the operation, reduce the operation difficulty, shorten the operation time and raise the operation precision.

This paper presents a novel augmented reality (AR)-based neurosurgical training simulator which provides a very natural way for surgeons to learn neurosurgical skills. Surgical simulation with bimanual haptic interaction is integrated in this work to provide a simulated environment for users to achieve holographic guidance for pre-operative training. To achieve the AR guidance, the simulator should precisely overlay the 3D anatomical information of the hidden target organs in the patients in real surgery. In this regard, the patient-specific anatomy structures are reconstructed from segmented brain magnetic resonance imaging. We propose a registration method for precise mapping of the virtual and real information. In addition, the simulator provides bimanual haptic interaction in a holographic environment to mimic real brain tumor resection. In this study, we conduct AR-based guidance validation and a user study on the developed simulator, which demonstrate the high accuracy of our AR-based neurosurgery simulator, as well as the AR guidance mode’s potential to improve neurosurgery by simplifying the operation, reducing the difficulty of the operation, shortening the operation time, and increasing the precision of the operation.