Ultrasound (US) imaging is widely employed for diagnosis and staging of vascular diseases, mainly due to its high availability and the fact it does not emit ionizing radiation. However, high interoperator variability limits the repeatability of US image acquisition. To address this challenge, we propose an end-to-end workflow for automatic robotic US screening of tubular structures using only real-time US imaging feedback. First, a U-Net was trained for real-time segmentation of vascular structure from cross-sectional US images. Then, we represented the detected vascular structure as a 3-D point cloud, which was used to estimate the centerline of the target structure and its local radius by solving a constrained nonlinear optimization problem. Iterating the previous processes, the US probe was automatically aligned to the normal direction of the target structure, while the object was constantly maintained in the center of the US view. The real-time segmentation result was evaluated both on a phantom and in vivo on brachial arteries of volunteers. In addition, the whole process was validated using both simulation and physical phantoms. The mean absolute orientation, centering, and radius error ($\pm$ SD) on a gel phantom were $\text{3.7}\pm \text{1.6}^{\circ }$, $0.2\pm \text{0.2}\,\text{mm}$ and $0.8\pm \text{0.4}\,\text{mm}$, respectively. The results indicate that the method can automatically screen tubular structures with an optimal probe orientation (i.e., normal to the vessel) and accurately estimate the radius of the target structure.

Researchers are developing augmented reality visualization systems to provide accessible and user-friendly interfaces for medical intervention and patient information systems. A related video can be seen here: http://youtu.be/kifj0ZP4Mos. It shows how augmented reality visualization systems can provide accessible and user-friendly interfaces for medical intervention and patient information systems.

Hybrid tracers that are both radioactive and fluorescent help extend the use of fluorescence-guided surgery to deeper structures. Such hybrid tracers facilitate preoperative surgical planning using (3D) scintigraphic images and enable synchronous intraoperative radio- and fluorescence guidance. Nevertheless, we previously found that improved orientation during laparoscopic surgery remains desirable. Here we illustrate how intraoperative navigation based on optical tracking of a fluorescence endoscope may help further improve the accuracy of hybrid surgical guidance. After feeding SPECT/CT images with an optical fiducial as a reference target to the navigation system, optical tracking could be used to position the tip of the fluorescence endoscope relative to the preoperative 3D imaging data. This hybrid navigation approach allowed us to accurately identify marker seeds in a phantom setup. The multispectral nature of the fluorescence endoscope enabled stepwise visualization of the two clinically approved fluorescent dyes, fluorescein and indocyanine green. In addition, the approach was used to navigate toward the prostate in a patient undergoing robot-assisted prostatectomy. Navigation of the tracked fluorescence endoscope toward the target identified on SPECT/CT resulted in real-time gradual visualization of the fluorescent signal in the prostate, thus providing an intraoperative confirmation of the navigation accuracy.

Thyroid volumetry is crucial in the diagnosis, treatment, and monitoring of thyroid diseases. However, conventional thyroid volumetry with 2D ultrasound is highly operator-dependent. This study compares 2D and tracked 3D ultrasound with an automatic thyroid segmentation based on a deep neural network regarding inter- and intraobserver variability, time, and accuracy. Volume reference was MRI. 28 healthy volunteers (24—50 a) were scanned with 2D and 3D ultrasound (and by MRI) by three physicians (MD 1, 2, 3) with different experience levels (6, 4, and 1 a). In the 2D scans, the thyroid lobe volumes were calculated with the ellipsoid formula. A convolutional deep neural network (CNN) automatically segmented the 3D thyroid lobes. 26, 6, and 6 random lobe scans were used for training, validation, and testing, respectively. On MRI (T1 VIBE sequence) the thyroid was manually segmented by an experienced MD. MRI thyroid volumes ranged from 2.8 to 16.7ml (mean 7.4, SD 3.05). The CNN was trained to obtain an average Dice score of 0.94. The interobserver variability comparing two MDs showed mean differences for 2D and 3D respectively of 0.58 to 0.52ml (MD1 vs. 2), −1.33 to −0.17ml (MD1 vs. 3) and −1.89 to −0.70ml (MD2 vs. 3). Paired samples t-tests showed significant differences for 2D (p = .140, p = .002 and p = .002) and none for 3D (p = .176, p = .722 and p = .057). Intraobsever variability was similar for 2D and 3D ultrasound. Comparison of ultrasound volumes and MRI volumes showed a significant difference for the 2D volumetry of all MDs (p = .002, p = .009, p <.001), and no significant difference for 3D ultrasound (p = .292, p = .686, p = 0.091). Acquisition time was significantly shorter for 3D ultrasound. Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.

Molecular imaging is one of the pillars of precision surgery. Its applications range from early diagnostics to therapy planning, execution, and the accurate assessment of outcomes. In particular, molecular imaging solutions are in high demand in minimally invasive surgical strategies, such as the substantially increasing field of robotic surgery. This review aims at connecting the molecular imaging and nuclear medicine community to the rapidly expanding armory of surgical medical devices. Such devices entail technologies ranging from artificial intelligence and computer-aided visualization technologies (software) to innovative molecular imaging modalities and surgical navigation (hardware). We discuss technologies based on their role at different steps of the surgical workflow, i.e., from surgical decision and planning, over to target localization and excision guidance, all the way to (back table) surgical verification. This provides a glimpse of how innovations from the technology fields can realize an exciting future for the molecular imaging and surgery communities.