Ultrasound examination is widely used for diagnosis of breast cancer, which requires a full-coverage scan of the whole organ and stable probe–breast interaction for high-quality image acquisition. In this study, a robotic system for automated breast ultrasound scanning is proposed. To avoid occlusion, point cloud of the breast is obtained from multiple angles and registered together for accurate tissue shape reconstruction, then scan path planning is performed with isometric 3-D point cloud searching algorithm for full and uniform coverage. To reduce the adverse influence of large tissue deformation during ultrasound scanning, a contact force–strain regression model is built for tissue deformation estimation and used as a feedforward to correct the planned scanning path, and a probe–tissue interaction model with scanning resistance is built for normal probe orientation adjustment. To maintain normal probe orientation and stable contact force during robotic ultrasound scanning, a hybrid force–velocity control framework is proposed based on admittance control algorithm. Experiment results show that the robotic system can successfully maintain a constant contact force and keep the probe's posture along the normal orientation of probe–breast interaction both in in-plane direction and out-of-plane direction.

Natural orifice transluminal endoscopic surgery (NOTES) has received significant attentions due to its minimal incision trauma compared with traditional multi-port robot assisted surgery. Continuum robot can be used in NOTES due to its high flexibility which can adapt to circuitous paths. However, the modeling of continuum robot is complex due to its nonlinear dynamics, especially with uncertain external disturbance. In this letter a hybrid adaptive control framework is proposed, combining offline trained robot inverse kinematics with neural network and online adaptive adjustment of PID controller parameters with another neural network, providing additional driving amount to the robot system to compensate the positioning error caused by external disturbance. Path tracking and point tracking experiments with external load are conducted and the results validate that the proposed hybrid adaptive control framework can compensate uncertain factors such as friction, driving tendon relaxation and external load during robot movement, and obtain relatively high positioning accuracy (maximum absolute error less than 2 mm) with quickresponse.

Automatic report generation has arisen as a significant research area in computer-aided diagnosis, aiming to alleviate the burden on clinicians by generating reports automatically based on medical images. In this work, we propose a novel framework for automatic ultrasound report generation, leveraging a combination of unsupervised and supervised learning methods to aid the report generation process. Our framework incorporates unsupervised learning methods to extract potential knowledge from ultrasound text reports, serving as the prior information to guide the model in aligning visual and textual features, thereby addressing the challenge of feature discrepancy. Additionally, we design a global semantic comparison mechanism to enhance the performance of generating more comprehensive and accurate medical reports. To enable the implementation of ultrasound report generation, we constructed three large-scale ultrasound image-text datasets from different organs for training and validation purposes. Extensive evaluations with other state-of-the-art approaches exhibit its superior performance across all three datasets. Code and dataset are valuable at this link.

Autonomous ultrasound scanning robots have attracted the attention of researchers, and the real-time quality assessment of ultrasound images is the key technology of them. Existing robot systems usually use pixel-level feature statistical methods such as grayscale, confidence map, etc. However, in clinical practice doctors’ evaluation of ultrasound image quality not only relies on the pixel quality, but also on the image content. In this study, we introduced the deep learning method to the quality assessment of medical breast ultrasound images to learn the doctors’ clinical evaluation standards. We collected 1205 breast ultrasound images of 533 patients and asked experienced doctors to score them. The ResNet18 with a shallow number of layers is adopted to extract features of ultrasound images, and the high-order feature coding model BCNN (Bilinear CNN) in fine-grained categorization is adopted to improve the accuracy of quality assessment. The consistency between the ultrasound image quality assessment results of our method and the doctors’ annotation results reached 0.842 by PLCC (Pearson Linear Correlation Coefficient). Compared with the confidence map method commonly used for quality assessment in the automatic ultrasound scanning robots, our method achieves a higher consistency with the evaluation results of doctors and provides a new image evaluation method and idea for the visual servo control framework of ultrasound autonomous scanning robots.

In robot-assisted endoscopic nasal surgery, due to the narrow and complex nasal cavity and sinus anatomical structure, the nasal endoscope of robot end-effector is prone to injure surrounding tissues during surgical approaches. In order to improve the safety of endoscope movement in robot-assisted endoscopic nasal surgery, this paper proposes a path planning method for endoscopic surgical approaches based on medical image pixel map searching, and safe surgical paths from the nasal entrance point to the operating areas are obtained. Firstly, through the trajectory analysis of nasal endoscope tip during surgery, the path planning requirements of endoscope tip are analyzed and obtained. Then, considering the narrow and complex anatomy of the nasal cavity, a binarized three-dimensional grid map containing the spatial anatomy of the nasal cavity is constructed based on the patient's CT medical image sequence. The endoscopic surgical approaches are searched and optimized by introducing the A-star algorithm, and safe surgical paths from the nasal entrance point to the operating areas are obtained. Finally, a virtual nasal endoscopy system is developed and tested on a head model containing nasal tissue, and the effectiveness of the planned surgical paths is verified by automatic virtual nasal endoscopy browsing experiment.