Puncture robots pave a new way for stable, accurate and safe percutaneous liver tumor puncture operation. However, affected by respiratory motion, intraoperative accurate location of the tumor and its surrounding anatomical structures remains a difficult problem in existing robot-assisted puncture operations. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D ultrasound (US) and preoperative 3D computed tomography (CT) fusion is proposed, addressing the shortcomings of existing puncture robots. To deal with the challenge of cross-modal and cross-dimensional registration between 2D US and 3D CT, a decoupled two-stage registration approach combining initial vessel structure-based 3D US – 3D CT registration with intraoperative intensity-based 2D US -3D US registration is proposed. To achieve fast and robust ultrasound probe calibration, a method based on an improved N-wire phantom is proposed. Twenty puncture experiments are performed in different breath-holding positions on a respiratory motion simulation platform, and experimental results show that the mean puncture error is 2.48 mm, which can meet the requirements in a wide of clinical scenarios Note to Practitioners —In clinical percutaneous liver tumor puncture operation, due to the lack of real-time and clear image guidance, it is difficult to locate the tumor and its surrounding vital anatomical structures. In addition, the stability and accuracy of manual operation are poor. The development of a puncture robot is an effective solution for these problems. However, existing CT and magnetic resonance imaging (MRI) guided robots do not consider the tumor localization errors caused by inconsistent breath-holding positions between preoperative scan period and intraoperative puncture period, and US guided robots are limited by the poor image quality and the narrow field of vision. In this paper, a dual-arm robotic needle insertion system with guidance of intraoperative 2D US and preoperative 3D CT fusion is proposed. This system can take advantage of the real-time ultrasound and clear CT images at the same time, and can provide real-time, clear and all-round guidance for percutaneous liver tumor puncture operation, which has obvious advantages over the existing puncture robots. Phantom experiments have been completed and animal experiments will be carried out in the future.

Percutaneous needle insertion is a widely used technique in needle-based intervention; however, it is difficult to achieve high targeting accuracy due to needle deflection and the boundary effect in multilayered tissues. This paper presents a preoperative needle insertion trajectory planning method that includes needle deflection modeling, insertion angle correction and needle trajectory optimization. The needle deflection model can predict the needle deflection with preoperatively obtained information, which includes tissue-specific parameters and the geometric/material properties of the needle. An insertion angle shift distribution model is obtained based on experimental data, and is used to correct insertion angles at tissue boundaries. The needle trajectory optimization is based on the weighted sum of cost function and balances the assessment criteria, which include the trajectory length, the insertion angle, and the distance to critical structures, and determines the needle trajectory with the best performance. Experiments are conducted to demonstrate that with the proposed preoperative needle insertion trajectory planning method, a needle insertion trajectory with increased accuracy, desired insertion length and safe distance to critical structures can be obtained.

Needle insertion is widely used in modern clinical practice, and flexible needles enable surgeons to adjust the needle insertion path to avoid obstacles and compensate for target drift. To increase the accuracy and safety of the needle insertion procedure, a path planning and tracking control method for flexible needle steering in dynamic environment is proposed in this paper. The path planning method is based on the artificial potential field algorithm and curve fitting algorithm, after several feasible paths with variant curvatures is obtained, they are furtherly optimized based on path length, obstacle clearance and control effort to select the best one. Simulations show that the proposed path planning method performs better than the current constant curvature method and the rapidly-exploring random tree (RRT) based method. By finely tuning the initial planned path, an intraoperative path replanning method is proposed, which are used to compensate for the drift of target and obstacles. Then to ensure the needle is inserted along the planned path, a tracking control method is adopted based on fuzzy logic algorithm. With the proposed intraoperative path planning and tracking control method, the robustness and accuracy of needle insertion in congested environment can be improved, and experiments also show that our proposed method achieves satisfactory insertion safety and targeting accuracy in dynamic environment. The obtained accuracy of 1.57 mm is sufficient for most clinical needle insertion applications.

As an effective and efficient discriminative learning method, the broad learning system (BLS) has received increasing attention due to its outstanding performance without large computational resources. The standard BLS is derived under the minimum mean square error (MMSE) criterion, while MMSE is with poor performance when dealing with imbalanced data. However, imbalanced data are widely encountered in real-world applications. To address this issue, a novel cost-sensitive BLS algorithm (CS-BLS) is proposed. In the CS-BLS, many variations can be adopted, and CS-BLS with weighted cross-entropy is analyzed in this paper. Weighted penalty factors are used in CS-BLS to constrain the contribution of each sample in different classes. The samples in minor classes are allocated higher weights to increase their contributions. Four different weight calculation methods are adopted to the CS-BLS, and thus, four CS-BLS methods are proposed: Log-CS-BLS, Lin-CS-BLS, Sqr-CS-BLS, and EN-CS-BLS. Experiments based on artificially imbalanced datasets of MNIST and small NORB are firstly conducted and compared with the standard BLS. The results show that the proposed CS-BLS methods have better generalization and robustness than the standard BLS. Then, experiments on a real ultrasound breast image dataset are conducted, and the results demonstrate that the proposed CS-BLS methods are effective in actual medical diagnosis.