The unequal distribution of medical resources and scarcity of experienced practitioners confine access to bronchoscopy primarily to well-equipped hospitals in developed regions, contributing to the unavailability of bronchoscopic services in underdeveloped areas. Here, we present an artificial intelligence (AI) co-pilot bronchoscope robot that empowers novice doctors to conduct lung examinations as safely and adeptly as experienced colleagues. The system features a user-friendly, plug-and-play catheter, devised for robot-assisted steering, facilitating access to bronchi beyond the fifth generation in average adult patients. Drawing upon historical bronchoscopic videos and expert imitation, our AI–human shared control algorithm enables novice doctors to achieve safe steering in the lung, mitigating misoperations. Both in vitro and in vivo results underscore that our system equips novice doctors with the skills to perform lung examinations as expertly as seasoned practitioners. This study offers innovative strategies to address the pressing issue of medical resource disparities through AI assistance.

Continuum robots, owing to their inherent compliance, have become essential in endoscopic surgical procedures, such as mucosal ablation. However, the prevalent design of endoscopic manipulators, which typically features only a single active bending segment, often results in limited dexterity and accessibility. Additionally, the incorporation of variable stiffness in these robots has attracted significant interest, with the aim to improve manipulation capabilities in confined spaces. In the paper, we propose a novel variable-length continuum robot with variable stiffness for endoscopic surgery. The robot’s stiffness can be altered either by modifying the catheter’s length or solid-liquid transition of low-melting-point alloy (LMPA). The design and fabrication methods of the robot are meticulously detailed. Additionally, a quasi-static stiffness model along with a learning-based stiffness compensation approach for accurate stiffness estimation are proposed. Leveraging this model, a contact force controller is designed for ablation procedure. The experimental results show that our robot possesses good flexibility and accessibility, making it highly adept at manipulating in confined spaces. Its variable stiffness feature significantly enhances its ability to counteract external disturbance and prevent tip deformation (with a average position change of 1.1mm). Finally, through force control experiments and a surgical demonstration in a gastrointestinal model, we have further validated the robot’s applicability in surgical contexts. Note to Practitioners —This paper proposed a variable-length continuum robot with variable stiffness for endoscopic surgery. The robot can achieve axial elongation and omnidirectional bending motion, having better dexterity and accessibility than traditional medical continuum robots with one active bending segment. The robot’s stiffness can be adjusted by the length changes or solid-liquid transition of low-melting-point alloy (LMPA). Besides, an accurate stiffness model and a contact force controller are proposed for endoscopic ablation surgery. By experimental results, the robot shows high flexibility and accessibility, allowing access to confined spaces for manipulation, and good control accuracy and variable stiffness capability for endoscopic surgery.

Bronchial foreign body aspiration is a life-threatening condition with a high incidence across diverse populations, requiring urgent diagnosis and treatment. However, the limited availability of skilled practitioners and advanced medical equipment in community clinics and underdeveloped regions underscores the broader challenges in emergency care. Here, we present a cost-effective robotic bronchoscope capable of computed tomography (CT)–free, artificial intelligence (AI)–driven foreign body search and doctor-collaborated removal over long distances via fifth-generation (5G) communication. The system is built around a low-cost (<5000 USD), portable (<2 kilograms) bronchoscope robotic platform equipped with a 3.3-millimeter-diameter catheter and 1-millimeter biopsy forceps designed for safe pulmonary search and foreign body removal. Our AI algorithm, which integrates classical data structures with modern machine learning techniques, enables thorough CT-free lung coverage. The tree structure is leveraged to memorize a compact exploration process and guide the decision-making. Both virtual and physical simulations demonstrate the system’s effective autonomous foreign body search, minimizing bronchial wall contact to reduce patient discomfort. In a remote procedure, a physician in Hangzhou successfully retrieved a foreign body from a live pig located 1500 kilometers away in Chengdu using 5G communication, highlighting effective collaboration of AI, robotics, and human experts. We anticipate that this 5G-enabled, low-cost, AI expert–collaborated robotic platform has notable potential to reduce medical disparities, enhance emergency care, improve patient outcomes, decrease physician workload, and streamline medical procedures through the automation of routine tasks.

Intra-operative target pose estimation is fundamental in minimally invasive surgery (MIS) to guiding surgical robots. This task can be fulfilled by the 2-D/3-D rigid registration, which aligns the anatomical structures between intra-operative 2-D fluoroscopy and the pre-operative 3-D computed tomography (CT) with annotated target information. Although this technique has been researched for decades, it is still challenging to achieve accuracy, robustness and efficiency simultaneously. In this paper, a novel orthogonal-view 2-D/3-D rigid registration framework is proposed which combines the dense reconstruction based on deep learning and the GPU-accelerated 3-D/3-D rigid registration. First, we employ the X2CT-GAN to reconstruct a target CT from two orthogonal fluoroscopy images. After that, the generated target CT and pre-operative CT are input into the 3-D/3-D rigid registration part, which potentially needs a few iterations to converge the global optima. For further efficiency improvement, we make the 3-D/3-D registration algorithm parallel and apply a GPU to accelerate this part. For evaluation, a novel tool is employed to preprocess the public head CT dataset CQ500 and a CT-DRR dataset is presented as the benchmark. The proposed method achieves 1.65 ± 1.41 mm in mean target registration error(mTRE), 20% in the gross failure rate(GFR) and 1.8 s in running time. Our method outperforms the state-of-the-art methods in most test cases. It is promising to apply the proposed method in localization and nano manipulation of micro surgical robot for highly precise MIS.

Common medical image segmentation tasks require large training datasets with pixel-level annotations which are very expensive and time-consuming to prepare. To overcome such limitation and achieve the desired segmentation accuracy, a novel Weakly-Interactive-Mixed Learning (WIML) framework is proposed by efficiently using weak labels. On one hand, utilize weak labels to reduce annotation time for high-quality strong labels by designing a Weakly-Interactive Annotation (WIA) part of the WIML which prudently introduces interactive learning into the weakly-supervised segmentation strategy. On the other hand, utilize weak labels and very few strong labels to achieve desired segmentation accuracy by designing a Mixed-Supervised Learning (MSL) part of the WIML which can boost the segmentation accuracy by providing strong prior knowledge during training. Besides, a multi-task Full-Parameter-Sharing Network (FPSNet) is proposed to better implement this framework. Specifically, to further reduce annotation time, attention modules (scSE) are integrated into FPSNet to improve the class activation map (CAM) performance for the first time. To further improve segmentation accuracy, a Full-Parameter-Sharing (FPS) strategy is designed in FPSNet to alleviate the overfitting of the segmentation task supervised by very few strong labels. The proposed method is validated on the BraTS 2019 and LiTS 2017 datasets, and experiments demonstrate that the proposed method WIML-FPSNet outperforms several state-of-the-art segmentation methods with minimal annotation efforts. Our code is made publicly available at: https://github.com/NieXiuping/WIML

Lung nodule detection and segmentation from computed tomography (CT) scans are essential for lung cancer diagnosis and therapy. Inspired by the substantial progress of neural networks in 2D natural images, current methods often adapt 2D networks to 3D for these tasks, resulting in massive network parameters. However, the limited training data available for lung nodule CT scans (≈2 k) compared to natural images (>10 b) lead to overfitting and reduced performance. This study proposes a lightweight architecture for lung nodule detection and segmentation, alleviating the overfitting caused by the data scarcity. To reduce the parameters of the backbone network, the deformable Transformer is introduced to retain large receptive fields for desired performance with a small model size. To eliminate extensive anchor‐related parameters, this study employs the anchor‐free structure for the detection task, which also aligns the common features with the segmentation task. Benefiting from this, a higher level of parameter sharing becomes possible between two tasks, further reducing the entire model size and imposing a strong regularizer to alleviate the overfitting. The proposed method is validated on the LUNA16 and LIDC‐IDRI datasets, and experimental results demonstrate that the method outperforms the state‐of‐the‐art methods with a small model size. Code: https://github.com/LiuLiluZJU/LWNet.