Ultrasound examination is widely used for diagnosis of breast cancer, which requires a full-coverage scan of the whole organ and stable probe–breast interaction for high-quality image acquisition. In this study, a robotic system for automated breast ultrasound scanning is proposed. To avoid occlusion, point cloud of the breast is obtained from multiple angles and registered together for accurate tissue shape reconstruction, then scan path planning is performed with isometric 3-D point cloud searching algorithm for full and uniform coverage. To reduce the adverse influence of large tissue deformation during ultrasound scanning, a contact force–strain regression model is built for tissue deformation estimation and used as a feedforward to correct the planned scanning path, and a probe–tissue interaction model with scanning resistance is built for normal probe orientation adjustment. To maintain normal probe orientation and stable contact force during robotic ultrasound scanning, a hybrid force–velocity control framework is proposed based on admittance control algorithm. Experiment results show that the robotic system can successfully maintain a constant contact force and keep the probe's posture along the normal orientation of probe–breast interaction both in in-plane direction and out-of-plane direction.

Natural orifice transluminal endoscopic surgery (NOTES) has received significant attentions due to its minimal incision trauma compared with traditional multi-port robot assisted surgery. Continuum robot can be used in NOTES due to its high flexibility which can adapt to circuitous paths. However, the modeling of continuum robot is complex due to its nonlinear dynamics, especially with uncertain external disturbance. In this letter a hybrid adaptive control framework is proposed, combining offline trained robot inverse kinematics with neural network and online adaptive adjustment of PID controller parameters with another neural network, providing additional driving amount to the robot system to compensate the positioning error caused by external disturbance. Path tracking and point tracking experiments with external load are conducted and the results validate that the proposed hybrid adaptive control framework can compensate uncertain factors such as friction, driving tendon relaxation and external load during robot movement, and obtain relatively high positioning accuracy (maximum absolute error less than 2 mm) with quickresponse.

Autonomous ultrasound scanning robots have attracted the attention of researchers, and the real-time quality assessment of ultrasound images is the key technology of them. Existing robot systems usually use pixel-level feature statistical methods such as grayscale, confidence map, etc. However, in clinical practice doctors’ evaluation of ultrasound image quality not only relies on the pixel quality, but also on the image content. In this study, we introduced the deep learning method to the quality assessment of medical breast ultrasound images to learn the doctors’ clinical evaluation standards. We collected 1205 breast ultrasound images of 533 patients and asked experienced doctors to score them. The ResNet18 with a shallow number of layers is adopted to extract features of ultrasound images, and the high-order feature coding model BCNN (Bilinear CNN) in fine-grained categorization is adopted to improve the accuracy of quality assessment. The consistency between the ultrasound image quality assessment results of our method and the doctors’ annotation results reached 0.842 by PLCC (Pearson Linear Correlation Coefficient). Compared with the confidence map method commonly used for quality assessment in the automatic ultrasound scanning robots, our method achieves a higher consistency with the evaluation results of doctors and provides a new image evaluation method and idea for the visual servo control framework of ultrasound autonomous scanning robots.

In robot-assisted endoscopic nasal surgery, due to the narrow and complex nasal cavity and sinus anatomical structure, the nasal endoscope of robot end-effector is prone to injure surrounding tissues during surgical approaches. In order to improve the safety of endoscope movement in robot-assisted endoscopic nasal surgery, this paper proposes a path planning method for endoscopic surgical approaches based on medical image pixel map searching, and safe surgical paths from the nasal entrance point to the operating areas are obtained. Firstly, through the trajectory analysis of nasal endoscope tip during surgery, the path planning requirements of endoscope tip are analyzed and obtained. Then, considering the narrow and complex anatomy of the nasal cavity, a binarized three-dimensional grid map containing the spatial anatomy of the nasal cavity is constructed based on the patient's CT medical image sequence. The endoscopic surgical approaches are searched and optimized by introducing the A-star algorithm, and safe surgical paths from the nasal entrance point to the operating areas are obtained. Finally, a virtual nasal endoscopy system is developed and tested on a head model containing nasal tissue, and the effectiveness of the planned surgical paths is verified by automatic virtual nasal endoscopy browsing experiment.

Puncture robots pave a new way for stable, accurate and safe percutaneous liver tumor puncture operation. However, affected by respiratory motion, intraoperative accurate location of the tumor and its surrounding anatomical structures remains a difficult problem in existing robot-assisted puncture operations. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D ultrasound (US) and preoperative 3D computed tomography (CT) fusion is proposed, addressing the shortcomings of existing puncture robots. To deal with the challenge of cross-modal and cross-dimensional registration between 2D US and 3D CT, a decoupled two-stage registration approach combining initial vessel structure-based 3D US – 3D CT registration with intraoperative intensity-based 2D US -3D US registration is proposed. To achieve fast and robust ultrasound probe calibration, a method based on an improved N-wire phantom is proposed. Twenty puncture experiments are performed in different breath-holding positions on a respiratory motion simulation platform, and experimental results show that the mean puncture error is 2.48 mm, which can meet the requirements in a wide of clinical scenarios Note to Practitioners —In clinical percutaneous liver tumor puncture operation, due to the lack of real-time and clear image guidance, it is difficult to locate the tumor and its surrounding vital anatomical structures. In addition, the stability and accuracy of manual operation are poor. The development of a puncture robot is an effective solution for these problems. However, existing CT and magnetic resonance imaging (MRI) guided robots do not consider the tumor localization errors caused by inconsistent breath-holding positions between preoperative scan period and intraoperative puncture period, and US guided robots are limited by the poor image quality and the narrow field of vision. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D US and preoperative 3D CT fusion is proposed. This system can take advantage of the real-time ultrasound and clear CT images at the same time, and can provide real-time, clear and all-round guidance for percutaneous liver tumor puncture operation, which has obvious advantages over the existing puncture robots. Phantom experiments have been completed and animal experiments will be carried out in the future.