Multi-dimensional force sensors have substantially improved the surgical quality of surgical robots. The purpose of this paper is to provide a comprehensive review of the literature on the design and application of multi-dimensional force sensors in various surgical robots. The multi-dimensional force sensor can monitor the interaction force between the surgical instrument and the soft tissue, enabling doctors to obtain force feedback information in real time during surgery. This can help them improve the accuracy and safety of surgery. The review identifies the importance of sensors classification and performance parameters in surgery for multi-dimensional force measurement. This review focuses on the integrated design of force sensors and multi-dimensional force decoupling algorithms. In addition, a fiber bragg grating multi-dimensional force sensor for puncture surgery was designed to validate the effectiveness of the proposed method. This review paper provides a comprehensive classification and analysis of multi-dimensional force sensors for surgical robots, which will help researchers to understand the integrated design solutions for different surgical instruments and multi-dimensional force sensors.

In medical puncture robots, visible light, infrared and ultrasound images are currently used to guide punctures. The lack of information about the interaction forces between the puncture needle and soft tissue in different directions during the puncture process can easily lead to soft tissue being damaged. The current three-dimensional force sensors are large and can only be mounted on the base of the puncture needle, which does not allow for easy integration. Moreover, the force transfer to the base introduces various disturbing forces and the measurement accuracy is low. To reduce the risk of soft tissue being damaged and to enhance the intelligent control strategy of the puncture robot, this paper designs a three-dimensional force sensor based on fiber Bragg gratings. The sensor is very small and can be integrated into the back end of the puncture needle to accurately measure the interaction forces between the puncture needle and the soft tissue in different directions. The puncture needle wall is designed with notched bending of a multilayer continuous beam, which can increase the sensitivity of axial stiffness, while maintaining the sensitivity of the sensor to lateral bending and torsion, and also reduce the crosstalk between the axial and lateral forces. The finite element method is used to optimize its structural parameters, and a BP neural network based on the global optimal fitness function is proposed to solve the decoupling problem between the three-dimensional forces, which effectively improves the detection accuracy of the force sensor. The experimental results show that the measurement error of the sensor is less than 1.5%, which can accurately measure the interaction force between the puncture needle and the soft tissue and improve the safety of the puncture process.

Background: Various surgical skills lead to differences in patient outcomes and identifying poorly skilled surgeons with constructive feedback contributes to surgical quality improvement. The aim of the study was to develop an algorithm for evaluating surgical skills in laparoscopic cholecystectomy based on the features of elementary functional surgical gestures (Surgestures). Materials and methods: Seventy-five laparoscopic cholecystectomy videos were collected from 33 surgeons in five hospitals. The phase of mobilization hepatocystic triangle and gallbladder dissection from the liver bed of each video were annotated with 14 Surgestures. The videos were grouped into competent and incompetent based on the quantiles of modified global operative assessment of laparoscopic skills (mGOALS). Surgeon-related information, clinical data, and intraoperative events were analyzed. Sixty-three Surgesture features were extracted to develop the surgical skill classification algorithm. The area under the receiver operating characteristic curve of the classification and the top features were evaluated. Results: Correlation analysis revealed that most perioperative factors had no significant correlation with mGOALS scores. The incompetent group has a higher probability of cholecystic vascular injury compared to the competent group (30.8 vs 6.1%, P=0.004). The competent group demonstrated fewer inefficient Surgestures, lower shift frequency, and a larger dissection-exposure ratio of Surgestures during the procedure. The area under the receiver operating characteristic curve of the classification algorithm achieved 0.866. Different Surgesture features contributed variably to overall performance and specific skill items. Conclusion: The computer algorithm accurately classified surgeons with different skill levels using objective Surgesture features, adding insight into designing automatic laparoscopic surgical skill assessment tools with technical feedback.

The inertial measurement unit (IMU)-to-segment (I2S) alignment is an important part of IMU-based joint angle estimation, and the accurate estimation of the three degree of freedom (3-DOF) knee angle can provide practical support for the evaluation of motions. In this paper, we introduce a dynamic weight particle swarm optimization (DPSO) algorithm with crossover factor based on the joint constraint to obtain the dynamic alignment vectors of I2S, and use them to perform the quaternion-based 3-DOF knee angle estimation algorithm. The optimization algorithm and the joint angle estimation algorithm were evaluated by comparing with the optical motion capture system. The range of 3-DOF knee angle root mean square errors (RMSEs) is 1.6°–5.9° during different motions. Furthermore, we also set up experiments of human walking (3 km/h), jogging (6 km/h) and ordinary running (9 km/h) to investigate the effects of dynamic I2S misalignment errors on 3-DOF knee angle estimation during different motions by artificially adding errors to I2S alignment parameters. The results showed differences in the effects of I2S misalignment errors on the estimation of knee abduction, internal rotation and flexion, which indicate the differences in knee joint kinematics among different motions. The IMU to thigh misalignment error has the greatest effect on the estimation of knee internal rotation. The effect of IMU to thigh misalignment error on the estimation of knee abduction angle becomes smaller and then larger during the two processes of switching from walking to jogging and then speeding up to ordinary running. The effect of IMU to shank misalignment error on the estimation of knee flexion angle is numerically the largest, while the standard deviation (SD) is the smallest. This study can provide support for future research on the accuracy of 3-DOF knee angle estimation during different motions.

Kinematic analysis and simulation of minimally invasive surgical robot (MISR) systems are critical for the control system design with remote center of motion (RCM) constraint. In this paper, a MISR system using bimanual manipulator is introduced. The system is simulated under a virtual reality (VR) environment to validate the dynamics and control. A new control scheme was proposed to track desired position and orientation of the surgery instrument under the RCM constraint. The VR system has been utilized to compare different inverse kinematics solver to improve the computation efficiency. Both the simulation and the experiment results show that the new control scheme provide fast online solution for the bimanual MISR system, which could also be extended to other manipulation applications.

Accurate and robust segmentation of surgical instrument parts is urgently required by AI enhanced intelligent surgery and automatic surgery skill evaluation. However, it is still a challenging problem due to the multiple small areas of frontal clasper and wrist parts, complicated object occlusion and light variation issues. Increasing high quality labeled data is effective to improve the performance but labor-costing. A novel instrument data augmentation method is proposed to generate high quality training data through planar instrument replacement and orientation adjustment. Moreover, image repainting methods are utilized to recover the occluded parts by instruments and generate clean tissue images. Then joint angle reconfigured instruments are placed on recovered clean tissue images to generate simulated surgery images, and we control the occlusion rates and overlapping areas by which high quality instrument manipulation augmented training data are generated automatically. Detailed experiment results show our proposed data augmentation method make existing methods achieve better segmentation accuracy and robustness of wrist and clasper part, especially the fault detection errors decreased significantly in instrument overlapping and complicated lighting conditions.