17 January 2024 in IEEE/ASME Transactions on Mechatronics
The profile estimation for continuum robots is a crucial problem concerning automatically controlling robots. The conventional method is based on the Cosserat rod theory, which is limited by the dependence of the convergence on the initial guess and computational complexity. To tackle these issues, this article proposes a general kinetostatic model to estimate the profile of the tendon-driven continuum robot (TDCR). We first abstract the backbone of the TDCR as an Euler–Bernoulli beam and then derive the spatial beam constraint model of a circular cross-section beam without considering torsion and shear. Next, taking a single-section TDCR as an example, we provide comprehensive modeling, considering the driving tendon tensions, friction, gravity, and external forces. Subsequently, an algorithm based on the chained spatial beam constraint model is proposed to estimate the robot's profile. The method can be generalized to the TDCR with different configurations. Simulations demonstrate the accuracy, computational efficiency, and computational success rate of our method, as well as its advantages over the state-of-the-art. Real-world experiments have also been performed to validate the effectiveness of our method with three different configurations of the TDCR.
22 June 2023 in IEEE Transactions on Instrumentation and Measurement
Micro force sensing in various clinical scenarios is a challenging issue to be addressed. It is highly difficult to trade off the size, cost, and measurement accuracy of a micro force sensing system. In this paper, a compact and affordable micro force sensing system enhanced by deep neural network is proposed. A three-axis force sensor is designed and fabricated with a footprint of only 14 mm and is employed to transform the force on the material structure into more accurate distance information. Such a sensor configuration can be seamlessly interfaced with the distal end of our in-house compliant and flexible continuum robot. On top of that, the RNN-LSTM network is exploited to augment the micro force sensing capability of the distal end-effector of the robot, which addresses the limitation on the nonlinear force issue of the continuum robot and the material itself. The RNN-LSTM network alone can be employed to perform force curve fitting for specific interventional tasks. The results indicate that more than 90% accuracy has been achieved, and the network can be applied to large-scale continuum robot-assisted interventional scenario deployment and teleoperation force perception.
19 February 2025 in IEEE/ASME Transactions on Mechatronics
The potential of continuum robots in medical applications is considerable, due to their flexibility and capacity to navigate complex anatomical environments. This article introduces a novel framework based on the Fast-Adaptive Permanent Magnetic Tracking method, which has been designed with the objective of enhancing the accuracy and autonomy of colonoscopic biopsies. The system incorporates a permanent magnet positioning methodology, enabling the robot to maintain a trajectory tracking root mean square error of less than 4 mm, with magnet speeds up to 150 mm/s and positioning errors under 2 mm. Furthermore, the framework includes an adaptive obstacle avoidance strategy, allowing the robot to navigate around obstacles and adjust its posture in response to dynamic movement. Extensive experimental validations in both simulation and real-world environments demonstrate the system's effectiveness in delivering precise, responsive, and continuous operation. This work represents a significant advancement in autonomous navigation based on permanent magnetic localization techniques, with the potential to enhance the efficacy and safety of robotic-assisted surgeries.
16 January 2025 in IEEE Transactions on Automation Science and Engineering
Deep learning can predict depth maps and capsule ego-motion from capsule endoscopy videos, aiding in 3D scene reconstruction and lesion localization. However, the collisions of the capsule endoscopies within the gastrointestinal tract cause vibration perturbations in the training data. Existing solutions focus solely on vision-based processing, neglecting other auxiliary signals like vibrations that could reduce noise and improve performance. Therefore, we propose V2-SfMLearner, a multimodal approach integrating vibration signals into vision-based depth and capsule motion estimation for monocular capsule endoscopy. We construct a multimodal capsule endoscopy dataset containing vibration and visual signals, and our artificial intelligence solution develops an unsupervised method using vision-vibration signals, effectively eliminating vibration perturbations through multimodal learning. Specifically, we carefully design a vibration network branch and a Fourier fusion module, to detect and mitigate vibration noises. The fusion framework is compatible with popular vision-only algorithms. Extensive validation on the multimodal dataset demonstrates superior performance and robustness against vision-only algorithms. Without the need for large external equipment, our V2-SfMLearner has the potential for integration into clinical capsule robots, providing real-time and dependable digestive examination tools. The findings show promise for practical implementation in clinical settings, enhancing the diagnostic capabilities of doctors.
23 October 2023 in IEEE Sensors Journal
Considering that Coupled Dictionary Learning (CDL) method can obtain a reasonable linear mathematical relationship between resource images, we propose a novel CDL-based Synthetic Aperture Radar (SAR) and multispectral pseudo-color fusion method. Firstly, the traditional Brovey transform is employed as a pre-processing method on the paired SAR and multispectral images. Then, CDL is used to capture the correlation between the pre-processed image pairs based on the dictionaries generated from the source images via enforced joint sparse coding. Afterward, the joint sparse representation in the pair of dictionaries is utilized to construct an image mask via calculating the reconstruction errors, and therefore generate the final fusion image. The experimental verification results of the SAR images from the Sentinel-1 satellite and the multispectral images from the Landsat-8 satellite show that the proposed method can achieve superior visual effects, and excellent quantitative indicators in terms of spectral distortion, correlation coefficient, MSE, NIQE, BRISQUE, and PIQE.
19 September 2024 in International Journal of Non-Linear Mechanics