25 April 2022 in IEEE Transactions on Medical Robotics and Bionics
Surgical instrument segmentation and depth estimation are crucial steps to improve autonomy in robotic surgery. Most recent works treat these problems separately, making the deployment challenging. In this paper, we propose a unified framework for depth estimation and surgical tool segmentation in laparoscopic images. The network has an encoder-decoder architecture and comprises two branches for simultaneously performing depth estimation and segmentation. To train the network end to end, we propose a new multi-task loss function that effectively learns to estimate depth in an unsupervised manner, while requiring only semi-ground truth for surgical tool segmentation. We conducted extensive experiments on different datasets to validate these findings. The results showed that the end-to-end network successfully improved the state-of-the-art for both tasks while reducing the complexity during their deployment.
13 December 2021 in Neural Computing and Applications
Human emotion recognition is an active research area in artificial intelligence and has made substantial progress over the past few years. Many recent works mainly focus on facial regions to infer human affection, while the surrounding context information is not effectively utilized. In this paper, we proposed a new deep network to effectively recognize human emotions using a novel global-local attention mechanism. Our network is designed to extract features from both facial and context regions independently, then learn them together using the attention module. In this way, both the facial and contextual information is used to infer human emotions, therefore enhancing the discrimination of the classifier. The intensive experiments show that our method surpasses the current state-of-the-art methods on recent emotion datasets by a fair margin. Qualitatively, our global-local attention module can extract more meaningful attention maps than previous methods. The source code and trained model of our network are available at https://github.com/minhnhatvt/glamor-net.
6 January 2022 in IEEE Transactions on Medical Imaging
Deformable registration is a crucial step in many medical procedures such as image-guided surgery and radiation therapy. Most recent learning-based methods focus on improving the accuracy by optimizing the non-linear spatial correspondence between the input images. Therefore, these methods are computationally expensive and require modern graphic cards for real-time deployment. In this paper, we introduce a new Light-weight Deformable Registration network that significantly reduces the computational cost while achieving competitive accuracy. In particular, we propose a new adversarial learning with distilling knowledge algorithm that successfully leverages meaningful information from the effective but expensive teacher network to the student network. We design the student network such as it is light-weight and well suitable for deployment on a typical CPU. The extensively experimental results on different public datasets show that our proposed method achieves state-of-the-art accuracy while significantly faster than recent methods. We further show that the use of our adversarial learning algorithm is essential for a time-efficiency deformable registration method. Finally, our source code and trained models are available at https://github.com/aioz-ai/LDR ALDK.
5 December 2023 in ACM Transactions on Graphics
Music-driven group choreography poses a considerable challenge but holds significant potential for a wide range of industrial applications. The ability to generate synchronized and visually appealing group dance motions that are aligned with music opens up opportunities in many fields such as entertainment, advertising, and virtual performances. However, most of the recent works are not able to generate high-fidelity long-term motions, or fail to enable controllable experience. In this work, we aim to address the demand for high-quality and customizable group dance generation by effectively governing the consistency and diversity of group choreographies. In particular, we utilize a diffusion-based generative approach to enable the synthesis of flexible number of dancers and long-term group dances, while ensuring coherence to the input music. Ultimately, we introduce a Group Contrastive Diffusion (GCD) strategy to enhance the connection between dancers and their group, presenting the ability to control the consistency or diversity level of the synthesized group animation via the classifier-guidance sampling technique. Through intensive experiments and evaluation, we demonstrate the effectiveness of our approach in producing visually captivating and consistent group dance motions. The experimental results show the capability of our method to achieve the desired levels of consistency and diversity, while maintaining the overall quality of the generated group choreography.
5 July 2024 in IEEE Transactions on Medical Robotics and Bionics
Autonomous robots in endovascular operations have the potential to navigate circulatory systems safely and reliably while decreasing the susceptibility to human errors. However, there are numerous challenges involved with the process of training such robots, such as long training duration and safety issues arising from the interaction between the catheter and the aorta. Recently, endovascular simulators have been employed for medical training but generally do not conform to autonomous catheterization due to the lack of standardization and RL framework compliance. Furthermore, most current simulators are closed-source, which hinders the collaborative development of safe and reliable autonomous systems through shared learning and community-driven enhancements. In this work, we introduce CathSim, an open-source simulation environment that accelerates the development of machine learning algorithms for autonomous endovascular navigation. We first simulate the high-fidelity catheter and aorta with a state-of-the-art endovascular robot. We then provide the capability of real-time force sensing between the catheter and the aorta in simulation. Furthermore, we validate our simulator by conducting two different catheterization tasks using two popular reinforcement learning algorithms, namely SAC and PPO. The experimental results show that our open-source simulator can mimic the behavior of real-world endovascular robots and facilitate the development of different autonomous catheterization tasks. Our simulator is publicly available at https://github.com/airvlab/cathsim.
8 December 2022 in IEEE Transactions on Biomedical Engineering
Objective: In-vivo validation on animal setting of a pneumatically propelled robot for endovascular intervention, to determine safety and clinical advantage of robotic cannulations compared to manual operation. Methods: Robotic assistance and image-guided intervention are increasingly used for improving endovascular procedures with enhanced navigation dexterity and accuracy. However, most platforms developed in the past decade still present inherent limitations in terms of altered clinical workflow, counterintuitive human-robot interaction, and a lack of versatility. We have created a versatile, highly integrated platform for robot-assisted endovascular intervention aimed at addressing such limitations, and here we demonstrate its clinical usability through in-vivo animal trials. A detailed in-vivo study on four porcine models conducted with our robotic platform is reported, involving cannulation and balloon angioplasty of five target arteries. Results: The trials showed a 100% success rate, and post-mortem histopathological assessment demonstrated a reduction in the incidence and severity of vessel trauma with robotic navigation versus manual manipulation. Conclusion: In-vivo experiments demonstrated that the applicability of our robotic system within the context of this study was well tolerated, with good feasibility, and low risk profile. Comparable results were observed with robotics and manual cannulation, with clinical outcome potentially in favor of robotics. Significance: This study showed that the proposed robotic platform can potentially improve the execution of endovascular procedures, paving the way for clinical translation.