In this article, a deep learning method for the shape sensing of continuum robots based on multicore fiber bragg grating (FBG) fiber is introduced. The proposed method, based on an artificial neural network (ANN), differs from traditional approaches, where accurate shape reconstruction requires a tedious characterization of many characteristic parameters. A further limitation of traditional approaches is that they require either multiple fibers, whose location relative to the centerline must be precisely known (calibrated), or a single multicore fiber whose position typically coincides with the neutral line. The proposed method addresses this limitation and, thus, allows shape sensing based on a single multicore fiber placed off-center. This helps in miniaturizing and leaves the central channel available for other purposes. The proposed approach was compared to a recent state-of-the-art model-based shape sensing approach. A two-degree-of-freedom benchtop fluidics-driven catheter system was built to validate the proposed ANN. The proposed ANN-based shape sensing approach was evaluated on a 40-mm-long steerable continuum robot in both 3-D free-space and 2-D constrained environments, yielding an average shape sensing error of 0.24 and 0.49 mm, respectively. With these results, the superiority of the proposed approach compared to the recent model-based shape sensing method was demonstrated.

In cardiovascular interventions, when steering catheters and especially robotic catheters, great care should be paid to prevent applying too large forces on the vessel walls as this could dislodge calcifications, induce scars or even cause perforation. To address this challenge, this paper presents a novel compliant motion control algorithm that relies solely on position sensing of the catheter tip and knowledge of the catheter’s behavior. The proposed algorithm features a data-driven tip position controller. The controller is trained based on a so-called control Long Short-Term Memory Network (control-LSTM). Trajectory following experiments are conducted to validate the quality of the proposed control-LSTM. Results demonstrated superior positioning capability with sub-degree precision of the new approach in the presence of severe rate-dependent hysteresis. Experiments both in a simplified setup as well as in an aortic phantom further show that the proposed approach allows reducing the interaction forces with the environment by around 70%. This work shows how deep learning can be exploited advantageously to avoid tedious modeling that would be needed to precisely steer continuum robots in constrained environments such as the patient’s vasculature.

Endovascular catheterization is an intervention which offers a low risk alternative to open surgery in many patients. Today’s interventions rely heavily on fluoroscopic imaging to guide interventionalists. Fluoroscopy only produces 2D visualization of the catheter and also exposes both the patient and interventionalists to harmful radiation. Different approaches have been proposed to overcome the limitations of fluoroscopy. Fiber Bragg Grating (FBG)-based shape sensing is becoming popular to reconstruct the catheter shape. Multi-core fibers with parallel optical cores are interesting as they allow 3D shape reconstruction with a single fiber. A common issue with FBG-based shape sensing is its sensitivity to variations in twist. Even small amounts of twist can significantly impact the overall shape reconstruction accuracy. This work proposes a novel approach which combines electromagnetic tracking (EMT), FBG-based shape sensing, and sparse fluoroscopic images. The method provides real-time 3D visualization of the catheter without the need of continuous fluoroscopy. A unique feature of the proposed method is the selective use of imaging for dynamic twist-compensation of the FBG sensor. The proposed sensor-fusion method improved 3D reconstruction accuracy. Real-world in-vitro experiments promising results. For a catheter with an embedded fiber length of 170 mm, the proposed approach the 3D shape with a median root-mean-square (rms) error of 0.39 mm and an interquartile range of 0.10 mm in the 2D experiment in which the catheter was bent in a plane. A median rms error of 0.54 mm and an interquartile range of 0.07 mm were achieved in the 3D experiments.

Continuum robots such as robotic catheters are increasingly being used in minimally invasive surgery. Compliance contributes to enhanced safety during e.g. catheter insertion, however, estimation of contact force and location may help clinicians avoiding exerting excessive force. Ultimately this could lead to faster and safer interventions. Researchers proposed force sensors integrated in the catheter tip in the past. However, such sensors add extra complexity to the catheter design. Also, tip force sensors do not provide insights on forces that act along the catheter length. This paper proposes a data-driven approach for localizing contact forces that appear over the length of the catheter. The proposed approach consists of a collision detection method and a contact localization method. The framework only requires the measurement of the catheter’s shape which can be done by an embedded multi-core Fiber Bragg Grating fiber. The method was validated experimentally with a 3D-printed continuum robot with an integrated multi-core fiber. A second contact localization method which is based on identifying the discontinuity in the measured curvature, is also implemented and compared with the proposed method. The static and dynamic experiments show a mean average localization error of 2.3 mm and 4.3 mm which correspond to respectively 3.3% and 6.1% of a 70 mm long flexible robot. These findings demonstrate that the proposed framework outperforms the previous methods and yield promising results. The contact state estimation algorithm can detect collisions in at most approximately $\text{1.08}\,\text{s}$.

Nowadays, navigating therapeutic catheters takes place under 2D fluoroscopic imaging. This requires considerable training of the clinician and exposes him/her to X-ray radiation. Researchers have increasingly investigated alternative sensing techniques. In this respect Fiber Bragg Grating (FBG)-based shape sensing is gaining popularity. This paper proposes two approaches to fuse FBG with fluoroscropy, improve the understanding of the 3- dimensional shape while reducing fluoroscopy use. This paper proposes 2 FBG-fluoroscopy fusion approaches that combine fluoroscopy and FBG measurements. A comparison is performed between 3D shape reconstructions based on biplane fluoroscopy, rigidly fused multi-core FBG and dynamically fused FBG shape reconstruction. To verify the performance of the different approaches experiments were performed with custom made catheter in a CathLab on 3D printed tubes with known ground truth shape. The experiments showed overall acceptable errors for the targeted application with a maximum below 2 mm. The error of shape reconstruction through biplane fluoroscopy, rigid and dynamic fusion were found to be 1.51+-0.04 mm, 1.77+-0.29 mm and 1.47+-0.15 mm, respectively. Thus, FBG-fluoroscopy fusion offers comparable results to fluoroscopy and may substantially reduce the radiation dose through optimal acquisition frequency.

Optical fiber-based shape sensing is gaining popularity in cardiac catheterization lately. Typically, these procedures are taking place under the guidance of fluoroscopy. However, fluoroscopy has several disadvantages. Thanks to fiber optic shape sensing and Electromagnetic Tracking (EMT), the 3D catheter shape can now be tracked in real-time without the need for fluoroscopy. Traditional optical fiber and EMT-based shape tracking methods have the drawback of the highest shape sensing error at the tip. The information offered by the EMT sensors is used mainly to localize the estimated shape in a fixed coordinate frame. In this letter, a novel approach for tracking the catheter is introduced to address the aforementioned problem. The catheter shape is directly reconstructed in the EMT coordinate frame by approximating the catheter shape by a number of Bézier curves while taking into account the curvatures measured by the optical fiber. Both 2D and 3D shape sensing experiments are conducted. The results of the 3D experiment show that the proposed method reduces the mean shape tracking error by approximately 38% (from 12.1 mm to 5.4 mm for a sensed length of 540 mm long) compared to the traditional method where the same number of sensors are used.