Deep learning has provided new ways of manipulating, processing and analyzing data. It sometimes may achieve results comparable to, or surpassing human expert performance, and has become a source of inspiration in the era of artificial intelligence. Another subfield of machine learning named reinforcement learning, tries to find an optimal behavior strategy through interactions with the environment. Combining deep learning and reinforcement learning permits resolving critical issues relative to the dimensionality and scalability of data in tasks with sparse reward signals, such as robotic manipulation and control tasks, that neither method permits resolving when applied on its own. In this paper, we present recent significant progress of deep reinforcement learning algorithms, which try to tackle the problems for the application in the domain of robotic manipulation control, such as sample efficiency and generalization. Despite these continuous improvements, currently, the challenges of learning robust and versatile manipulation skills for robots with deep reinforcement learning are still far from being resolved for real-world applications.

Objective: Minimally invasive surgical interventions in the gastrointestinal tract, such as endoscopic submucosal dissection (ESD), are very difficult for surgeons when performed with standard flexible endoscopes. Robotic flexible systems have been identified as a solution to improve manipulation. However, only a few such systems have been brought to preclinical trials as of now. As a result, novel robotic tools are required. Methods: We developed a telemanipulated robotic device, called STRAS, which aims to assist surgeons during intraluminal surgical endoscopy. This is a modular system, based on a flexible endoscope and flexible instruments, which provides 10 degrees of freedom (DoFs). The modularity allows the user to easily set up the robot and to navigate toward the operating area. The robot can then be teleoperated using master interfaces specifically designed to intuitively control all available DoFs. STRAS capabilities have been tested in laboratory conditions and during preclinical experiments. Results: We report 12 colorectal ESDs performed in pigs, in which large lesions were successfully removed. Dissection speeds are compared with those obtained in similar conditions with the manual Anubiscope platform from Karl Storz. We show significant improvements (p= 0.01). Conclusion: These experiments show that STRAS (v2) provides sufficient DoFs, workspace, and force to perform ESD, that it allows a single surgeon to perform all the surgical tasks and those performances are improved with respect to manual systems. Significance: The concepts developed for STRAS are validated and could bring new tools for surgeons to improve comfort, ease, and performances for intraluminal surgical endoscopy.

Optical coherence tomography (OCT) endoscopic catheters provide efficient solutions for the non-invasive scan of malignant tissues in internal human organs. In this letter, we investigate the image-guided control of a robotic flexible endoscopic system equipped with an OCT probe for autonomous tissue scanning. The visual control strategy is achieved by combining the image feedback from both a monocular endoscopic camera (eye-to-hand) and an OCT probe (eye-in-hand). Our control strategy allows the OCT probe to automatically track a trajectory defined by the user on the 2D endoscopic image and keep contact with the scanned surface. The orientation, depth, and endoscopic 2D image position are controlled by solving an optimization problem, which is converted to a quadratic programming problem. For the implementation of the visual control strategy, we also consider visibility constraints and actuators limitations. In addition, a marker-based method to estimate the 3D pose of the continuum robot using only 2D images is proposed based on a Kalman filter and a registration technique. The proposed control strategy is validated using both simulation and laboratory experiments.

Focused ultrasound (FUS) is a non-invasive and highly promising method for targeted and reversible blood-brain barrier permeabilization. Numerous preclinical studies aim to optimize the localized delivery of drugs using this method in rodents and non-human primates. Several clinical trials have been initiated to treat various brain diseases in humans using simultaneous BBB permeabilization and drug injection. This review presents the state of the art of in vitro and in vivo cavitation control algorithms for BBB permeabilization using microbubbles (MB) and FUS. Firstly, we describe the different cavitation states, their physical significance in terms of MB behavior and their translation into the spectral composition of the backscattered signal. Next, we report the different indexes calculated and used during the ultrasonic monitoring of cavitation. Finally, the different in vitro and in vivo cavitation control strategies described in the literature are presented and compared.

Surgical tool segmentation is a challenging and crucial task for computer and robot-assisted surgery. Supervised learning approaches have shown great success for this task. However, they need a large number of paired training data. Based on Generative Adversarial Networks (GAN), unpaired image-to-image translation (I2I) techniques (like CycleGAN and dualGAN) have been proposed to avoid the requirement of paired data and have been employed for surgical tool segmentation. The unpaired I2I methods avoid annotating images for domain changes. Instead of using them directly for the segmentation task, we propose new GAN-based methods for unpaired I2I by embedding a specific constraint for segmentation, namely each pixel of input image belongs to either background or surgical tool. Our methods simplify the architectures of existing unpaired I2I with a reduced number of generators and discriminators. Compared with dualGAN, the proposed strategies have a faster training process without reducing the accuracy of the segmentation. Besides, we show that, using textured tool images as annotated samples to train discriminators, unpaired I2I (including our methods) can achieve simultaneous tool image segmentation and repair (such as reflection removal and tool inpainting). The proposed strategies are validated for image segmentation of a flexible tool and for in vivo images from the EndoVis dataset.