3D point cloud registration is a fundamental problem in computer vision and robotics. Recently, learning-based point cloud registration methods have made great progress. However, these methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matching-based framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by a correspondence-based solver. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on object-level and scene-level benchmark datasets show that the proposed method achieves state-of-the-art performance. The code is available at: https://github.com/fukexue/RGM .

In image-guided neurosurgery, brain tissue displacement and deformation during neurosurgical procedures are a major source of error. In this paper, we implement and evaluate a linear-elastic-model-based framework for correction of brain shift using clinical data from five brain tumor patients. The framework uses a linear elastic model to simulate brain-shift behavior. The model is driven by cortical surface deformations, which are tracked using a surface-tracking algorithm combined with a laser-range scanner. The framework performance was evaluated using displacements of anatomical landmarks, tumor contours and self-defined evaluation parameters. The results show that tumor deformations predicted by the present framework agreed well with the ones observed intraoperatively, especially in the parts of the larger deformations. On average, a brain shift of 3.9 mm and a tumor margin shift of 4.2 mm were corrected to 1.2 and 1.3 mm, respectively. The entire correction process was performed in less than 5 min. The data from this study suggest that the technique is a suitable candidate for intraoperative brain-deformation correction.

Precise delineation of multiple organs or abnormal regions in the human body from medical images plays an essential role in computer-aided diagnosis, surgical simulation, image-guided interventions, and especially in radiotherapy treatment planning. Thus, it is of great significance to explore automatic segmentation approaches, among which deep learning-based approaches have evolved rapidly and witnessed remarkable progress in multi-organ segmentation. However, obtaining an appropriately sized and fine-grained annotated dataset of multiple organs is extremely hard and expensive. Such scarce annotation limits the development of high-performance multi-organ segmentation models but promotes many annotation-efficient learning paradigms. Among these, studies on transfer learning leveraging external datasets, semi-supervised learning including unannotated datasets and partially-supervised learning integrating partially-labeled datasets have led the dominant way to break such dilemmas in multi-organ segmentation. We first review the fully supervised method, then present a comprehensive and systematic elaboration of the 3 abovementioned learning paradigms in the context of multi-organ segmentation from both technical and methodological perspectives, and finally summarize their challenges and future trends.

Brain retraction causes great distortion that limits the accuracy of an image-guided neurosurgery system that uses preoperative images. Therefore, brain retraction correction is an important intraoperative clinical application. We used a linear elastic biomechanical model, which deforms based on the eXtended Finite Element Method (XFEM) within a framework for brain retraction correction. In particular, a laser range scanner was introduced to obtain a surface point cloud of the exposed surgical field including retractors inserted into the brain. A brain retraction surface tracking algorithm converted these point clouds into boundary conditions applied to XFEM modeling that drive brain deformation. To test the framework, we performed a brain phantom experiment involving the retraction of tissue. Pairs of the modified Hausdorff distance between Canny edges extracted from model-updated images, pre-retraction, and post-retraction CT images were compared to evaluate the morphological alignment of our framework. Furthermore, the measured displacements of beads embedded in the brain phantom and the predicted ones were compared to evaluate numerical performance. The modified Hausdorff distance of 19 pairs of images decreased from 1.10 to 0.76 mm. The forecast error of 23 stainless steel beads in the phantom was between 0 and 1.73 mm (mean 1.19 mm). The correction accuracy varied between 52.8 and 100 % (mean 81.4 %). The results demonstrate that the brain retraction compensation can be incorporated intraoperatively into the model-updating process in image-guided neurosurgery systems.