8 May 2023 in IEEE Journal of Biomedical and Health Informatics
Distributed big data and digital healthcare technologies have great potential to promote medical services, but challenges arise when it comes to learning predictive model from diverse and complex e-health datasets. Federated Learning (FL), as a collaborative machine learning technique, aims to address the challenges by learning a joint predictive model across multi-site clients, especially for distributed medical institutions or hospitals. However, most existing FL methods assume that clients possess fully labeled data for training, which is often not the case in e-health datasets due to high labeling costs or expertise requirement. Therefore, this work proposes a novel and feasible approach to learn a Federated Semi-Supervised Learning (FSSL) model from distributed medical image domains, where a federated pseudo-labeling strategy for unlabeled clients is developed based on the embedded knowledge learned from labeled clients. This greatly mitigates the annotation deficiency at unlabeled clients and leads to a cost-effective and efficient medical image analysis tool. We demonstrated the effectiveness of our method by achieving significant improvements compared to the state-of-the-art in both fundus image and prostate MRI segmentation tasks, resulting in the highest Dice scores of 89.23 and 91.95 respectively even with only a few labeled clients participating in model training. This reveals the superiority of our method for practical deployment, ultimately facilitating the wider use of FL in healthcare and leading to better patient outcomes.
23 March 2021 in IEEE Robotics and Automation Letters
This letter presents a miniature manipulator with variable stiffness towards minimally invasive transluminal endoscopic surgery, such as the endoscopic submucosal dissection (ESD). The manipulator comprises hollow modules with holes in the sidewall, compact with a 4 mm diameter, and dexterous with six degrees of freedom (DOFs). Basic operations of grasping and stripping can be achieved via the terminal surgical tools. The flexible tubes and torque coils are adopted to avoid the motion coupling of the joints. The worm gears with an anti-backdrive property are used to transfer power from the motor to the reels. The variable stiffness is achieved by using the spring-sliding block-based driver system, delivering both high payload/accuracy and flexibility. The system is described in detail. The kinematics, workspace and variable stiffness are analyzed. Tests in terms of the grasping force and variable stiffness are conducted. Results show that the grasping force is larger than 3 N, and the manipulator's stiffness can be tuned with high rates of change. The manipulator's performances, such as the dexterity and the coordination of operation under the master-slave configuration, were preliminarily demonstrated through the basic mock-up operations.
4 January 2023 in IEEE Transactions on Industrial Informatics
Traditional magnetic tracking approaches based on mathematical models and optimization algorithms are computationally intensive, depend on initial guesses, and do not guarantee convergence to a global optimum. Although fully-supervised data-driven deep learning can solve the above issues, the demand for a comprehensive dataset hampers its applicability in magnetic tracking. Thus, we propose an annular magnet pose estimation network (called AMagPoseNet) based on dual-domain few-shot learning from a prior mathematical model, which consists of two sub-networks: PoseNet and CaliNet. PoseNet learns to estimate the magnet pose from the prior mathematical model, and CaliNet is designed to narrow the gap between the mathematical model domain (MMD) and the real-world domain (RWD). Experimental results reveal that AMagPoseNet outperforms the optimization-based method regarding localization accuracy (1.87 $\pm$ 1.14 mm, 1.89 $\pm 0.81^{\circ }$ ), robustness (non-dependence on initial guesses), and computational latency (2.08 $\pm$ 0.02 ms). In addition, the six degrees of freedom (6-DoF) pose of the magnet could be estimated when discriminative magnetic field features are provided. With the assistance of the mathematical model, AMagPoseNet requires only a few real-world samples and has excellent performance, showing great potential for practical biomedical and industrial applications.
3 October 2023 in IEEE Transactions on Industrial Electronics
Robot-assisted gastrointestinal endoscopic surgery requires flexible manipulators to possess a compact dimension and stiffness tuning capability. Current stiffness-tunable miniature manipulators (STMM) using tendon-sheath mechanism (TSM) experience the problem of stiffness-influenced hysteresis. To address this, we propose the first stiffness-dependent M odified G eneralized P randtl– I shlinski (PI) model and a specific compensation strategy. First, we analyzed the stiffness tuning mechanism and extracted the stiffness parameter. Based on this, the analytical hysteresis model and its inverse function were built and verified in simulations and experiments, which increases the fitting accuracy of the hysteresis. Then, with the assistance of the compensation strategy, the real-time input–output relationship of the STMM's stiffness-tunable joints can be approximately linear. The average errors of trajectory tracking achieve a significant improvement of over 85%. This work provides a new method to model and compensate for the dynamic hysteresis in flexible endoscopic robots with the stiffness-tunable TSM.
18 February 2020 in IEEE/ASME Transactions on Mechatronics
This paper presents a robotic needle positioning approach with a novel 4-DoF parallel positioner, two back-driveable 2-channel linear/rotational driver, and a machine-body interface. Being more compact and flexible, Ni-Ti alloy wire-based Bowden cable transmission is introduced to connect the positioner and the drivers. For this new mechanism, forward and inverse kinematics are derived, and the workspace is analyzed. The mechanism's maximum incidence angle is 45°, and the stroke of the slider is 41 mm. The structural size of one positioner is ø70 mm×45 mm. The compact size and lightweight of the positioner makes it readily mountable on the skull or attached to other parts of the body. We fabricated a 3D printed proof-of-concept prototype of the system and tested its performance. The open-loop positioning accuracy of the slider and rotor is within ±1 mm and -0.8°to 1.5°, respectively. The trajectory tracking error of the system is within 1.71 mm. The stiffness of the positioner in x and y directions can be calculated as 9.03 N/mm and 11.91 N/mm, respectively. Finally, an image-guided navigation framework based on electromagnetic tracking demonstrates the feasibility of the proposed system on a phantom study.
19 September 2022 in IEEE Transactions on Automation Science and Engineering
An endoscope holder must understand the detailed surgical actions and the surgeons’ visual attention to keep important targets in the field of endoscopic view during operations. From an intensive analysis of the surgeons’ attention mechanism, we included that surgical actions, like cutting, suturing, etc., play an important role in determining the positions and weights of visual attention points during a dynamic surgical scene. To perform this process, this work proposes a Surgical Action-driven Visual Attention network (SAVAnet) and applies the network in autonomous endoscope control. Four scenarios are constructed in the da Vinci V-rep simulator: pick&place and needle exercise in a general laparoscopic training environment, needle driving with and without obstacle removal in an abdominal cavity, to create datasets for network training. The results show that the network has an outstanding performance in surgical action prediction with a high average accuracy of over 91%. Additionally, with surgical action guidance, the attention point prediction has higher accuracy and accords with surgeons’ visual attention. Finally, the acquired attention points are utilized to execute visual servoing in simulation. The results verify that the SAVAnet is feasible for autonomous endoscope control in real-time and lays a theoretical foundation for future sim-to-real execution. Note to Practitioners —This paper was motivated by the problem of endowing an endoscope with surgeons’ visual attention mechanism, which is affected by surgical actions, for autonomous endoscope control. An eye-tracking device has been utilized to detect surgeon’s visual attention in real-time and then control the endoscope to follow what the surgeon is looking at. However, this approach is susceptible to the surgical environment. Besides, many instrument detection and segmentation algorithms are developed for automatic surgical instrument tracking. However, surgeons’ visual attention does not always focus on the instruments during operations. In this work, we propose a novel SAVAnet to determine visual attention based on surgical actions. We prove from many qualitative and quantitative experiments that surgical actions play a significant role in determining visual attention. The designed SAVAnet can predict surgical actions correctly and then effectively guide the choice of visual attention. Finally, the simulation results show that the SAVAnet can endow endoscope with surgeons’ visual attention to perform self-control in real time. In future research, we will train the SAVAnet using real datasets and conduct more physical experiments on real surgical robots.