Ultrasound (US) imaging is widely employed for diagnosis and staging of vascular diseases, mainly due to its high availability and the fact it does not emit ionizing radiation. However, high interoperator variability limits the repeatability of US image acquisition. To address this challenge, we propose an end-to-end workflow for automatic robotic US screening of tubular structures using only real-time US imaging feedback. First, a U-Net was trained for real-time segmentation of vascular structure from cross-sectional US images. Then, we represented the detected vascular structure as a 3-D point cloud, which was used to estimate the centerline of the target structure and its local radius by solving a constrained nonlinear optimization problem. Iterating the previous processes, the US probe was automatically aligned to the normal direction of the target structure, while the object was constantly maintained in the center of the US view. The real-time segmentation result was evaluated both on a phantom and in vivo on brachial arteries of volunteers. In addition, the whole process was validated using both simulation and physical phantoms. The mean absolute orientation, centering, and radius error ($\pm$ SD) on a gel phantom were $\text{3.7}\pm \text{1.6}^{\circ }$, $0.2\pm \text{0.2}\,\text{mm}$ and $0.8\pm \text{0.4}\,\text{mm}$, respectively. The results indicate that the method can automatically screen tubular structures with an optimal probe orientation (i.e., normal to the vessel) and accurately estimate the radius of the target structure.

Acquiring good image quality is one of the main challenges for fully-automatic robot-assisted ultrasound systems (RUSS). The presented method aims at overcoming this challenge for orthopaedic applications by optimizing the orientation of the robotic ultrasound (US) probe, i.e. aligning the central axis of the US probe to the tissue's surface normal at the point of contact in order to improve sound propagation within the tissue. We first optimize the in-plane orientation of the probe by analyzing the confidence map [1] of the US image. We then carry out a fan motion and analyze the resulting forces estimated from joint torques to align the central axis of the probe to the normal within the plane orthogonal to the initial image plane. This results in the final 3D alignment of the probe's main axis with the normal to the anatomical surface at the point of contact without using external sensors for surface reconstruction or localizing the point of contact in an anatomical atlas. The algorithm is evaluated both on a phantom and on human tissues (forearm, upper arm and lower back). The mean absolute angular difference ( $\pm$ STD) between true and estimated normal on stationary phantom, forearm, upper arm and lower back was $3.1 \pm 1.0^\circ$ , $3.7 \pm 1.7^\circ$ , $5.3\pm 1.3^\circ$ and $6.9 \pm 3.5^\circ$ , respectively. In comparison, six human operators obtained errors of 3.2 \pm 1.7^\circ on the phantom. Hence the method is able to automatically position the probe normal to the scanned tissue at the point of contact and thus improve the quality of automatically acquired ultrasound images.

Subretinal injection is a delicate and complex microsurgery. The main surgical difficulties come from the surgeon's hand tremor, dexterous motion, and insufficient visual feedback. In order to begin addressing these challenges, this paper presents a robot system for subretinal insertion integrated with intraoperative optical coherence tomography (OCT). The surgical workflow using this system consists of two main parts: the first part is the manual robot control, which aims the target before approaching the retinal surface, while considering the remote center of motion (RCM) constraint. When the injection area has been located precisely, needle is inserted into retina. To ensure \revise{surgical safety}, needle insertion depth is estimated using OCT images on a continuous basis. A soft RCM control method is designed and integrated for the controller of our hybrid parallel-serial surgical robot. Safety and accuracy performance evaluation with a 15 ms control loop shows that the worst case RCM deviation error is within 1 mm. Experimental results demonstrated that the proposed system has the ability to improve surgical outcomes by surgeons overcome their physical limitations in order to enable a better dexterous motion, and furthermore enhancing their visual feedback for a better intraocular perception.

The correct orientation of an ultrasound (US) probe is one of the main parameters governing the US image quality. With the rise of robotic ultrasound systems (RUSS), methods that can automatically compute the orientation promise repeatable, automatic acquisition from predefined angles resulting in high-quality US imaging. In this paper, we propose a method to automatically position a US probe orthogonally to the tissue surface, thereby improving sound propagation and enabling RUSS to reach predefined orientations relatively to the surface normal at the contact point. The method relies on the derivation of the underlying mechanical model. Two rotations around orthogonal axes are carried out, while the contact force is being recorded. Then, the force data are fed into the model to estimate the normal direction. Accordingly, the probe orientation can be computed without requiring visual features. The method is applicable to the convex and linear probes. It has been evaluated on a phantom with varying tilt angles and on multiple human tissues (forearm, upper arm, lower back, leg). As a result, it has outperformed existing methods. The mean (±SD) absolute angular difference on in-vivo tissues averaged over all anatomies and probe types is 2.9±1.6 and 2.2±1.5 on the phantom.

With the advance in algorithms, deep reinforcement learning (DRL) offers solutions to trajectory planning under uncertain environments. Different from traditional trajectory planning which requires lots of effort to tackle complicated high-dimensional problems, the recently proposed DRL enables the robot manipulator to autonomously learn and discover optimal trajectory planning by interacting with the environment. In this article, we present state-of-the-art DRL-based collision-avoidance trajectory planning for uncertain environments such as a safe human coexistent environment. Since the robot manipulator operates in high dimensional continuous state-action spaces, model-free, policy gradient-based soft actor-critic (SAC), and deep deterministic policy gradient (DDPG) framework are adapted to our scenario for comparison. In order to assess our proposal, we simulate a 7-DOF Panda (Franka Emika) robot manipulator in the PyBullet physics engine and then evaluate its trajectory planning with reward, loss, safe rate, and accuracy. Finally, our final report shows the effectiveness of state-of-the-art DRL algorithms for trajectory planning under uncertain environments with zero collision after 5,000 episodes of training.

Tissue deformation in ultrasound (US) imaging leads to geometrical errors when measuring tissues due to the pressure exerted by probes. Such deformation has an even larger effect on 3D US volumes as the correct compounding is limited by the inconsistent location and geometry. This work proposes a patient-specified stiffness-based method to correct the tissue deformations in robotic 3D US acquisitions. To obtain the patient-specified model, robotic palpation is performed at sampling positions on the tissue. The contact force, US images and the probe poses of the palpation procedure are recorded. The contact force and the probe poses are used to estimate the nonlinear tissue stiffness. The images are fed to an optical flow algorithm to compute the pixel displacement. Then the pixel-wise tissue deformation under different forces is characterized by a coupled quadratic regression. In order to correct the deformation at unseen positions on the trajectory for building 3D volumes, an interpolation is performed based on the stiffness values computed at the sampling positions. With the stiffness and recorded force, the tissue displacement could be corrected. The method was validated on two blood vessel phantoms with different stiffness. The results demonstrate that the method is able to effectively correct the force-induced deformation and finally generate 3D tissue geometries.